-- Logs begin at Wed 2022-10-12 11:48:03 UTC, end at Wed 2022-10-12 12:59:13 UTC. -- Oct 12 12:19:08.138236 dv-cinder-812685-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 12 12:19:08.148466 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 12 12:19:08 2022] *** Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: nodename: dv-cinder-812685-16-iscsi Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: machine: x86_64 Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: clock source: unix Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: pcre jit disabled Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: detected number of CPU cores: 4 Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: current working directory: / Oct 12 12:19:08.150988 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: detected binary path: /usr/bin/uwsgi-core Oct 12 12:19:08.151557 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: your processes number limit is 47680 Oct 12 12:19:08.151557 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: your memory page size is 4096 bytes Oct 12 12:19:08.151557 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: detected max file descriptor number: 2048 Oct 12 12:19:08.151557 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: lock engine: pthread robust mutexes Oct 12 12:19:08.151557 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: thunder lock: enabled Oct 12 12:19:08.151557 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 12 12:19:08.151557 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 12 12:19:08.190183 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: Python main interpreter initialized at 0x55aa1ee4d9c0 Oct 12 12:19:08.190183 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: python threads support enabled Oct 12 12:19:08.190183 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: your server socket listen backlog is limited to 100 connections Oct 12 12:19:08.190183 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: your mercy for graceful operations on workers is 90 seconds Oct 12 12:19:08.190518 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: mapped 403077 bytes (393 KB) for 2 cores Oct 12 12:19:08.190617 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: *** Operational MODE: preforking *** Oct 12 12:19:08.190892 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: *** uWSGI is running in multiple interpreter mode *** Oct 12 12:19:08.190892 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: spawned uWSGI master process (pid: 81479) Oct 12 12:19:08.191084 dv-cinder-812685-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 12 12:19:08.194016 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: spawned uWSGI worker 1 (pid: 81490, cores: 1) Oct 12 12:19:08.194016 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: spawned uWSGI worker 2 (pid: 81491, cores: 1) Oct 12 12:19:08.194016 dv-cinder-812685-16-iscsi devstack@keystone.service[81479]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 12 12:19:12.490766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 12:19:12.490766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: from cryptography.utils import int_from_bytes Oct 12 12:19:12.491528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 12:19:12.491528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: from cryptography.utils import int_from_bytes Oct 12 12:19:12.496050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 12:19:12.496050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: from cryptography.utils import int_from_bytes Oct 12 12:19:12.496528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 12:19:12.496528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: from cryptography.utils import int_from_bytes Oct 12 12:19:12.547163 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 12:19:12.547360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 12:19:12.547557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] command line args: [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 12:19:12.547677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 12:19:12.547830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 12:19:12.547910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 12:19:12.548013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ================================================================================ {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 12:19:12.548105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] command line args: [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 12:19:12.548270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] admin_token = **** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.548353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 12:19:12.548469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ================================================================================ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 12:19:12.548546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] config_dir = [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.548692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.548779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] admin_token = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.548877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] config_source = [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.548954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] config_dir = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.549059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] debug = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.549144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.549277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.549542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] config_source = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.549542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] debug = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.549729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.549831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] list_limit = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] log_config_append = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.550913 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551112 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_config_append = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] log_dir = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] log_file = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_dir = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_file = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.551943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.552966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.553865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.556823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.556964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] publish_errors = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.557962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] publish_errors = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.558205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.558324 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.558444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.558444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.558738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.558738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.558957 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] standard_threads = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] standard_threads = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559806 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.559929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.560140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.560253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] use_journal = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.560365 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.560478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] use_json = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.560598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_journal = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.560712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] use_stderr = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.560824 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_json = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.560936 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] use_syslog = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.561054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_stderr = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.561204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.561356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] use_syslog = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.561356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:19:12.561571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.561684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.561802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.561916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.562175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.562293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.562407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.562520 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.562638 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.562751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.562872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.562992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.563106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.563218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.563331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.563449 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.563575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.563690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.563802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.563915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564261 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.564967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.565086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.565200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.565341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.565462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.565590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.565711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.565825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.565952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.566067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.566190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.566312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.566427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.566574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.566717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.566830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.566943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.567066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.567182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.567294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.567407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.567526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.567640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.567830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.567947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.568983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.569100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.569100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.569329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.569439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.569609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.569745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.569886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.570983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571076 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.571946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.572980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.573091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.573091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.573326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.573441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.573561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.573675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.573787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.573901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.574031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.574156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.574261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.574371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.574484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.574596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.574865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.574986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.575101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.575203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.575321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.575434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.575546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.575655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.575767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.575881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.576955 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.577079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.577193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.577312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.577432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.577545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.577658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.577782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.577902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.578945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.579063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.579251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.579374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.579488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.579614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.579729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.579845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.579967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.580087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.580202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.580315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.580427 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.580540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.580665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.580794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.580915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.581978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.582094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.582219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.582335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.582448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.582560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.582674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.582787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.582908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.583938 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584051 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.584981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.585096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.585209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.585321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.585436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.585548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.585669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.585790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.585790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.586101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.586199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.586295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.593942 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] auth.external = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.external = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594757 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] auth.password = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.594975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] auth.token = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595204 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.password = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] auth.token = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.595997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] credential.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.596950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.597904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.598961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] federation.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.599962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600112 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600686 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.600974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601547 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601618 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.601899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.602079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.602153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.602153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.602291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.602364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.602433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.602433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.602559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.603912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604060 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.604980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.605956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.606960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.607962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.608997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609681 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609824 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609895 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.609981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610343 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.610978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.611952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612517 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.612944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.613957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] resource.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.614971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615246 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.615935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] role.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] role.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] role.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.616959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.617114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] role.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.617195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.617343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.617424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.617593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.617682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.617837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.617917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.618114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.618207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.618384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.623994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624913 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.624995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.625997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] token.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.626963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.627949 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628438 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628783 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.628971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.629083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.connection = **** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.629150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.629210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.629279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.629340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.629429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630405 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630834 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.630951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.631090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.631181 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.631291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.631372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.631527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.631606 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.631762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.631840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.632934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.633011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.633107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 12:19:12.633186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.633388 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.633623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:19:12.633808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 12:19:12.666467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.666703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.666935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.667371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.667642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.667863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.682673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.682952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.710506 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.710734 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.710821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.711021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.774382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.775265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.823670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.823901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.824041 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.824276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994681 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.994987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.995118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.995200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:12.995332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.024206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.024318 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.024537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.025153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.025316 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.025400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.025464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.025598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.025682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.025814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.025898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.026049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.026133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.026262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.037188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.046080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.058038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.058270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.058363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.058495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.058604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.058722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.058807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.058946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.059028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.059162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.059260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.059380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.059460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.059597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:19:13.076725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.076933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 12:19:13.077057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.077273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 12:19:13.077360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.077455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 12:19:13.077629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.077714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.077834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 12:19:13.078000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.078149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.078268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.078419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.078530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.078688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.078952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.088925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.089052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.089206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.089289 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.089482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.089565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.093998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.094086 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.096692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.096788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.099846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.099958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.100139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.100224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.100426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.100509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.100716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.100800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.101013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.101108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.101280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.101368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.101554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.101636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.101833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.101915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.102134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.102229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.102412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.102494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.102709 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.102790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.124996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.125215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.126478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.126697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.126782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.126984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.131374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.131670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.131762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.131957 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.132041 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.132244 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.138752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.139049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.139139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.139366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.144643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.144948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.145033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.145246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.145330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.145543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.145640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.145828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.145912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.146129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.146212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.146405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.146505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.146683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.146798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.146980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.162542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.162828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.162916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.163174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.166699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.166971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.169908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.170205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.170332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.170508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.174390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.174612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.177496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.177674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.180719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.180917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.181017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.181220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.181305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.181569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.181652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.181868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.181984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.182179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.182271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.182464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.198269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.198490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.198579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.198788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.198888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.199060 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.199145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.199328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.205776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.206099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.207339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.207626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.208856 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.209137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.209220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.209479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.211796 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.212626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.212878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.213057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.213171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.213264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.213457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.213574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.213743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.214165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.224858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.228950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.229100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.229787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.241757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.242472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.245194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.245894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.248119 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.248754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.249581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.250259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.251061 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.251347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.251708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.251793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.251996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.252393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.258022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.258718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.260013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.260692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.262880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.263277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.263569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.263651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.263856 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.263994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.264284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.264566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.271553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.271852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.272136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.272287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.272580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.272873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.279036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.279274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.279560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.279732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:19:13.279842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.280020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.280113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.280304 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.280395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.280586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.280670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.280863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.280952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.281140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.281221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.281425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.281530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.281714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.281808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.282035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.282129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.282350 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.282445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.282626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.282710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.282932 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.283273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.283556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:19:13.316482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55aa1ee4d9c0 pid: 81490 (default app) Oct 12 12:19:13.317515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55aa1ee4d9c0 pid: 81491 (default app) Oct 12 12:19:13.319777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91276f7f-039a-442e-9c59-68cd47d2c660 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:13.320063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91276f7f-039a-442e-9c59-68cd47d2c660 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:13.320336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91276f7f-039a-442e-9c59-68cd47d2c660 None None] PATH_INFO: `/v3` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:13.325018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Wed Oct 12 12:19:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:20.749766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7535-655b-48b2-a9a9-7346df6185f7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:20.750304 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7535-655b-48b2-a9a9-7346df6185f7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:20.750386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e7535-655b-48b2-a9a9-7346df6185f7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:20.754595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:20.759746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:20.760180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:20.760273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:20.829745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: INFO dbcounter [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] Registered counter for database keystone Oct 12 12:19:20.849035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG oslo_db.sqlalchemy.engines [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81490) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 12:19:20.855900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writer thread running {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 12:19:21.033043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG passlib.handlers.bcrypt [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81490) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 12 12:19:21.033431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG passlib.handlers.bcrypt [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81490) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 12 12:19:21.091819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:21.093413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:21.219013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:19:21.666869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7535-655b-48b2-a9a9-7346df6185f7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:21.736329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: INFO dbcounter [None req-1a2e7535-655b-48b2-a9a9-7346df6185f7 None None] Registered counter for database keystone Oct 12 12:19:21.757694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG oslo_db.sqlalchemy.engines [None req-1a2e7535-655b-48b2-a9a9-7346df6185f7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81491) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 12:19:21.764496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writer thread running {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 12:19:21.881896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e7535-655b-48b2-a9a9-7346df6185f7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:21.886200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18289d12-7e69-4181-bf25-8526bdc1603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:21.886823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18289d12-7e69-4181-bf25-8526bdc1603b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:21.887101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18289d12-7e69-4181-bf25-8526bdc1603b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:21.887371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18289d12-7e69-4181-bf25-8526bdc1603b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:21.895460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18289d12-7e69-4181-bf25-8526bdc1603b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:21.946354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING keystone.common.rbac_enforcer.enforcer [None req-18289d12-7e69-4181-bf25-8526bdc1603b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 12 12:19:21.946646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18289d12-7e69-4181-bf25-8526bdc1603b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:21.955809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Wed Oct 12 12:19:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:19:21.961675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:21.979141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918108e-bb26-489a-83ee-58c54f807003 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:21.983605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02425f-5dd4-437d-ac10-e3740724d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:21.984697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02425f-5dd4-437d-ac10-e3740724d0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:21.984981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02425f-5dd4-437d-ac10-e3740724d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:21.985253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02425f-5dd4-437d-ac10-e3740724d0b6 admin admin] PATH_INFO: `/v3/projects/5655a305f69b4cb0901b114b25e5c478` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:21.986885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02425f-5dd4-437d-ac10-e3740724d0b6 admin admin] RBAC: Authorizing `identity:get_project(project_id=5655a305f69b4cb0901b114b25e5c478)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:22.039268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9f02425f-5dd4-437d-ac10-e3740724d0b6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 12 12:19:22.039616 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02425f-5dd4-437d-ac10-e3740724d0b6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:22.044675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Wed Oct 12 12:19:21 2022] GET /identity/v3/projects/5655a305f69b4cb0901b114b25e5c478 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:24.166712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21852834-281a-45fb-a202-50265d656ff7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:24.167223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21852834-281a-45fb-a202-50265d656ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:24.167328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21852834-281a-45fb-a202-50265d656ff7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:24.171316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:24.176659 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694533-a5d7-4a25-82a1-18327ec6786b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:24.177018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694533-a5d7-4a25-82a1-18327ec6786b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:24.177213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694533-a5d7-4a25-82a1-18327ec6786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:24.197488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c2694533-a5d7-4a25-82a1-18327ec6786b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:24.199031 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c2694533-a5d7-4a25-82a1-18327ec6786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:24.206842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:19:24.475456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21852834-281a-45fb-a202-50265d656ff7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:24.487879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21852834-281a-45fb-a202-50265d656ff7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:24.492049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fa580-5a9d-493e-8299-ca170d1936fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:24.492662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fa580-5a9d-493e-8299-ca170d1936fa admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:24.492941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fa580-5a9d-493e-8299-ca170d1936fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:24.493291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fa580-5a9d-493e-8299-ca170d1936fa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:24.501375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fa580-5a9d-493e-8299-ca170d1936fa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:24.503150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fa580-5a9d-493e-8299-ca170d1936fa admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:24.511706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Wed Oct 12 12:19:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:19:24.516880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694533-a5d7-4a25-82a1-18327ec6786b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:24.526460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694533-a5d7-4a25-82a1-18327ec6786b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:24.530801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9fedc-3a48-47dd-b73e-c7983796ac57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:24.531418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9fedc-3a48-47dd-b73e-c7983796ac57 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:24.531699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9fedc-3a48-47dd-b73e-c7983796ac57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:24.531974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9fedc-3a48-47dd-b73e-c7983796ac57 admin admin] PATH_INFO: `/v3/users/cd7cad0b678b4f0182359eae55570531` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:24.533287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9fedc-3a48-47dd-b73e-c7983796ac57 admin admin] RBAC: Authorizing `identity:get_user(user_id=cd7cad0b678b4f0182359eae55570531)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:24.535146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9fedc-3a48-47dd-b73e-c7983796ac57 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:24.536658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Wed Oct 12 12:19:24 2022] GET /identity/v3/users/cd7cad0b678b4f0182359eae55570531 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:26.711156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eca99-cbdc-44de-af83-e5d52c97c7de None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.711761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eca99-cbdc-44de-af83-e5d52c97c7de None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.711761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eca99-cbdc-44de-af83-e5d52c97c7de None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.713694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:26.718937 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a616-a599-41e0-b852-327bb34729eb None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.719385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a616-a599-41e0-b852-327bb34729eb None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.719605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a616-a599-41e0-b852-327bb34729eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.737590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2da0a616-a599-41e0-b852-327bb34729eb None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:26.738790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2da0a616-a599-41e0-b852-327bb34729eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:26.746822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133049-124f-4ae1-849b-53516c539ec4 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.747105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133049-124f-4ae1-849b-53516c539ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.747383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77133049-124f-4ae1-849b-53516c539ec4 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.748689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:26.753613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.753894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.754198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.795484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG passlib.handlers.bcrypt [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81491) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 12 12:19:26.795867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG passlib.handlers.bcrypt [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81491) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 12 12:19:26.819286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:26.845175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:26.846764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:26.947071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:19:27.086246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a616-a599-41e0-b852-327bb34729eb None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.111528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.189199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a616-a599-41e0-b852-327bb34729eb None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.210037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.228723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d9bdd-d950-4b23-9b03-fed1f56d4e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.229356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9bdd-d950-4b23-9b03-fed1f56d4e71 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.229633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9bdd-d950-4b23-9b03-fed1f56d4e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.229905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9bdd-d950-4b23-9b03-fed1f56d4e71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.240948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d9bdd-d950-4b23-9b03-fed1f56d4e71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-ef8d9bdd-d950-4b23-9b03-fed1f56d4e71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:27.250296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:27.251274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:27.252655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:27.257015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d9bdd-d950-4b23-9b03-fed1f56d4e71 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.310614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2929dc3-8e25-48ad-bea9-6502698d6b67 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.312912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.350047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.350200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.350200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.350200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.350200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.350494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.350791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.355207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d9bdd-d950-4b23-9b03-fed1f56d4e71 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.388239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.388835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.389118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.389387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.390201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:27.390434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.391074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.sql.core [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:27.391074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:19:27.391074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [parameters: {'id': 'f9f5ac5f724c4a93a809073df1dd4486', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 12:19:27.391074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:19:27.391802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.403495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 8/15] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:27.408885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:19:27.410228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 12:19:27.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application yield resource Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise self.value Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application session.commit() Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:19:27.412058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:19:27.413081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:19:27.414754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:19:27.416248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [parameters: {'id': 'f9f5ac5f724c4a93a809073df1dd4486', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 12:19:27.418115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:27.419165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 8/16] 10.222.0.44 () {60 vars in 1137 bytes} [Wed Oct 12 12:19:27 2022] POST /identity/v3/domains => generated 190 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:27.420233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.509942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4622c-6780-40f9-a6bd-776b6361c557 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.516455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300991d-d067-4fcf-836d-012fb7ebecfb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.544736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f5151-bf52-4c93-b7e3-d52a28bddf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.545355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f5151-bf52-4c93-b7e3-d52a28bddf7f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.545629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f5151-bf52-4c93-b7e3-d52a28bddf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.545898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f5151-bf52-4c93-b7e3-d52a28bddf7f None admin] PATH_INFO: `/v3/users/cd7cad0b678b4f0182359eae55570531` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.547055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f5151-bf52-4c93-b7e3-d52a28bddf7f None admin] RBAC: Authorizing `identity:get_user(user_id=cd7cad0b678b4f0182359eae55570531)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.548490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f5151-bf52-4c93-b7e3-d52a28bddf7f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.549637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.550047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 9/17] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/users/cd7cad0b678b4f0182359eae55570531 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:27.550520 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.550799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.551066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.555491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f5151-bf52-4c93-b7e3-d52a28bddf7f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.569166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.569562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.570671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:27.586596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:27.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:27.589068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:27.589068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:27.589068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:27.589068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 9/18] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:27.593874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.616079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.654257 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f5151-bf52-4c93-b7e3-d52a28bddf7f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.688450 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ab1ba-d90a-439b-8c6a-6abab9267db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.689084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ab1ba-d90a-439b-8c6a-6abab9267db0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.689360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ab1ba-d90a-439b-8c6a-6abab9267db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.689628 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ab1ba-d90a-439b-8c6a-6abab9267db0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.690772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ab1ba-d90a-439b-8c6a-6abab9267db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.692190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ab1ba-d90a-439b-8c6a-6abab9267db0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.693719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 10/19] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:27.696153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ebb24-0e66-4498-bccc-68ac0703830d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.699709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ab1ba-d90a-439b-8c6a-6abab9267db0 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.729731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24ce4a-b1c5-42e7-abed-622997163101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.730369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ce4a-b1c5-42e7-abed-622997163101 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.730658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ce4a-b1c5-42e7-abed-622997163101 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.730925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ce4a-b1c5-42e7-abed-622997163101 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.731571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ce4a-b1c5-42e7-abed-622997163101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:27.731873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ce4a-b1c5-42e7-abed-622997163101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.733243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ce4a-b1c5-42e7-abed-622997163101 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.750038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 10/20] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:27.783270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.802788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ab1ba-d90a-439b-8c6a-6abab9267db0 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.840344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f59a1-3cb3-4366-a758-168c30bcd241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.841015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f59a1-3cb3-4366-a758-168c30bcd241 None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.841296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f59a1-3cb3-4366-a758-168c30bcd241 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.841569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f59a1-3cb3-4366-a758-168c30bcd241 None admin] PATH_INFO: `/v3/domains/default/users/cd7cad0b678b4f0182359eae55570531/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.843551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f59a1-3cb3-4366-a758-168c30bcd241 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cd7cad0b678b4f0182359eae55570531, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.845021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f59a1-3cb3-4366-a758-168c30bcd241 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:28.045512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 11/21] 10.222.0.44 () {58 vars in 1361 bytes} [Wed Oct 12 12:19:27 2022] PUT /identity/v3/domains/default/users/cd7cad0b678b4f0182359eae55570531/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:19:30.464111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755bf93-6374-4d46-89b7-1675e6a0e1d2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.464631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755bf93-6374-4d46-89b7-1675e6a0e1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.464631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755bf93-6374-4d46-89b7-1675e6a0e1d2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.466363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 11/22] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:30.482896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9143-4316-4486-afad-d9b7d00c2d99 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.483411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9143-4316-4486-afad-d9b7d00c2d99 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.485841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9143-4316-4486-afad-d9b7d00c2d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.510180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-668f9143-4316-4486-afad-d9b7d00c2d99 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:30.511795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-668f9143-4316-4486-afad-d9b7d00c2d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:30.596610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a9f7-6e36-475c-a580-d0c63acf1cb1 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.596610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a9f7-6e36-475c-a580-d0c63acf1cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.596610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397a9f7-6e36-475c-a580-d0c63acf1cb1 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.598316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 12/23] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:30.605308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2055-1a52-4cd2-99be-46d95de19fcf None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.605595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2055-1a52-4cd2-99be-46d95de19fcf None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.605875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2055-1a52-4cd2-99be-46d95de19fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.632472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 12/24] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:30.643148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-514a2055-1a52-4cd2-99be-46d95de19fcf None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:30.644384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-514a2055-1a52-4cd2-99be-46d95de19fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:30.752630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 13/25] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:30.785450 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b46f0-edda-492f-8ec0-e2f4c549259d None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.785741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b46f0-edda-492f-8ec0-e2f4c549259d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.786049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b46f0-edda-492f-8ec0-e2f4c549259d None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.794931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 13/26] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:30.796827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705da217-faac-44f0-b06d-9786506ddf0c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.797120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705da217-faac-44f0-b06d-9786506ddf0c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.797422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705da217-faac-44f0-b06d-9786506ddf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.826956 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-705da217-faac-44f0-b06d-9786506ddf0c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:30.828163 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-705da217-faac-44f0-b06d-9786506ddf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:30.831103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:30.855466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ec181-a937-4d3c-845b-638fbb54663e None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.855963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ec181-a937-4d3c-845b-638fbb54663e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.856414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ec181-a937-4d3c-845b-638fbb54663e None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.857893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 14/27] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:30.869372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b2d3a-c340-4554-855c-7f99dacdf415 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.869824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b2d3a-c340-4554-855c-7f99dacdf415 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.870282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b2d3a-c340-4554-855c-7f99dacdf415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.907245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-882b2d3a-c340-4554-855c-7f99dacdf415 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:30.908733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-882b2d3a-c340-4554-855c-7f99dacdf415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:30.936133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 14/28] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:31.013315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:31.020494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 15/29] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:19:31.105731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705da217-faac-44f0-b06d-9786506ddf0c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.237684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705da217-faac-44f0-b06d-9786506ddf0c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.268998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b2d3a-c340-4554-855c-7f99dacdf415 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.285958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bdb7e-eb85-4845-803f-50b808a27060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.285958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bdb7e-eb85-4845-803f-50b808a27060 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.285958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bdb7e-eb85-4845-803f-50b808a27060 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.285958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bdb7e-eb85-4845-803f-50b808a27060 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.286352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bdb7e-eb85-4845-803f-50b808a27060 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.286352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bdb7e-eb85-4845-803f-50b808a27060 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.403164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b2d3a-c340-4554-855c-7f99dacdf415 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.439094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470e112-f41c-45fb-8db6-f4e5a792d9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.440028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470e112-f41c-45fb-8db6-f4e5a792d9bd None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.440530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470e112-f41c-45fb-8db6-f4e5a792d9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.441004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470e112-f41c-45fb-8db6-f4e5a792d9bd None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.441886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470e112-f41c-45fb-8db6-f4e5a792d9bd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.443654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470e112-f41c-45fb-8db6-f4e5a792d9bd None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.471363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:31.516484 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 16/30] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Oct 12 12:19:31 2022] POST /identity/v3/roles => generated 219 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:19:31.518226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470e112-f41c-45fb-8db6-f4e5a792d9bd None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.594363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:31.607780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 15/31] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Oct 12 12:19:31 2022] POST /identity/v3/roles => generated 225 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:31.611372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bdb7e-eb85-4845-803f-50b808a27060 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.688516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470e112-f41c-45fb-8db6-f4e5a792d9bd None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.771575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4200141-b7e5-4238-9232-8e03a299f5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.772580 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4200141-b7e5-4238-9232-8e03a299f5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.773077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4200141-b7e5-4238-9232-8e03a299f5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.773540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4200141-b7e5-4238-9232-8e03a299f5b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.784505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bdb7e-eb85-4845-803f-50b808a27060 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.806284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4200141-b7e5-4238-9232-8e03a299f5b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-d4200141-b7e5-4238-9232-8e03a299f5b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:31.811894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:31.812816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:31.813571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 17/32] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:31.822530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e31d-a5b2-47e3-a0bc-3842099f2909 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.824325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e31d-a5b2-47e3-a0bc-3842099f2909 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.825778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90e31d-a5b2-47e3-a0bc-3842099f2909 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.827601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 18/33] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:31.830043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90e31d-a5b2-47e3-a0bc-3842099f2909 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.836855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aeee63-635f-471b-b07a-c443c15e71ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.837888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeee63-635f-471b-b07a-c443c15e71ee None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.838464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeee63-635f-471b-b07a-c443c15e71ee None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.838948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aeee63-635f-471b-b07a-c443c15e71ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.860111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aeee63-635f-471b-b07a-c443c15e71ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.862096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aeee63-635f-471b-b07a-c443c15e71ee None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-95aeee63-635f-471b-b07a-c443c15e71ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:31.882398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:31.883284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:31.884206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:31.884206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:31.884206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:31.886128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 16/34] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:31.887490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb597-a6ff-4378-8b78-ad72a1309dff None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.888721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb597-a6ff-4378-8b78-ad72a1309dff None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.889220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb597-a6ff-4378-8b78-ad72a1309dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.908644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-934fb597-a6ff-4378-8b78-ad72a1309dff None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:31.910173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-934fb597-a6ff-4378-8b78-ad72a1309dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:31.967666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:31.968514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90e31d-a5b2-47e3-a0bc-3842099f2909 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.990548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 17/35] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:31.991425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb597-a6ff-4378-8b78-ad72a1309dff None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.996598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:32.014863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:32.015739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.016023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.019094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.019094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:32.019094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:32.019730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:32.031059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 19/36] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:32.036930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:32.139912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb597-a6ff-4378-8b78-ad72a1309dff None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:32.162813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d59437-6130-4e0e-bf49-f648eab18749 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:32.181294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:32.182355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.182798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.183214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.184057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:32.184510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:32.186206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:32.199693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 18/37] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:32.206444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:32.262279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:32.324313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fe7ad-02c4-41d7-9272-143b506f824f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:32.327152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf92c5-dc41-4837-9548-0f8ab3f8b844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:32.327824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf92c5-dc41-4837-9548-0f8ab3f8b844 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.328072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf92c5-dc41-4837-9548-0f8ab3f8b844 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.328351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf92c5-dc41-4837-9548-0f8ab3f8b844 None admin] PATH_INFO: `/v3/users/cd7cad0b678b4f0182359eae55570531` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.329496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf92c5-dc41-4837-9548-0f8ab3f8b844 None admin] RBAC: Authorizing `identity:get_user(user_id=cd7cad0b678b4f0182359eae55570531)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:32.330972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf92c5-dc41-4837-9548-0f8ab3f8b844 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:32.332584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 20/38] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:32 2022] GET /identity/v3/users/cd7cad0b678b4f0182359eae55570531 => generated 247 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:32.333339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf92c5-dc41-4837-9548-0f8ab3f8b844 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:32.363536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4eaf3-1215-443d-83eb-145f70497adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:32.364152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4eaf3-1215-443d-83eb-145f70497adf None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.364439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4eaf3-1215-443d-83eb-145f70497adf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.364929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4eaf3-1215-443d-83eb-145f70497adf None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.365373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4eaf3-1215-443d-83eb-145f70497adf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:32.366857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4eaf3-1215-443d-83eb-145f70497adf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:32.383497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:32.432565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf92c5-dc41-4837-9548-0f8ab3f8b844 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:32.465840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c40ce-dcc4-44b5-b957-447c8e67fb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:32.466474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c40ce-dcc4-44b5-b957-447c8e67fb80 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.466761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c40ce-dcc4-44b5-b957-447c8e67fb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.467038 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c40ce-dcc4-44b5-b957-447c8e67fb80 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.467716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c40ce-dcc4-44b5-b957-447c8e67fb80 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:32.469073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c40ce-dcc4-44b5-b957-447c8e67fb80 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:32.472024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 19/39] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:32 2022] POST /identity/v3/projects => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:32.474144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac302-5393-413e-813f-a44b44312827 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.475787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac302-5393-413e-813f-a44b44312827 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.475787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ac302-5393-413e-813f-a44b44312827 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.476009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 20/40] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:32.476795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ac302-5393-413e-813f-a44b44312827 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:32.558658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 21/41] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Oct 12 12:19:32 2022] POST /identity/v3/roles => generated 223 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:32.558658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1bb5-14ad-4c6f-af30-49cbd7f4bd4e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.558658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1bb5-14ad-4c6f-af30-49cbd7f4bd4e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.558658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1bb5-14ad-4c6f-af30-49cbd7f4bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.578496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a85b1bb5-14ad-4c6f-af30-49cbd7f4bd4e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:32.580056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a85b1bb5-14ad-4c6f-af30-49cbd7f4bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:32.595609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:32.608323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ac302-5393-413e-813f-a44b44312827 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:32.656284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527a564-c14f-4a0f-85ac-a8844d80ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:32.657236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527a564-c14f-4a0f-85ac-a8844d80ad54 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.657690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527a564-c14f-4a0f-85ac-a8844d80ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.658290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527a564-c14f-4a0f-85ac-a8844d80ad54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.659747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527a564-c14f-4a0f-85ac-a8844d80ad54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:32.661469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527a564-c14f-4a0f-85ac-a8844d80ad54 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:32.663582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 21/42] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:32.669800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527a564-c14f-4a0f-85ac-a8844d80ad54 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:32.685369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 22/43] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:32.787829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527a564-c14f-4a0f-85ac-a8844d80ad54 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:32.823568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:32.824182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.824465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.824740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.825504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.domain.id=default, user.id=cd7cad0b678b4f0182359eae55570531) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:32.825815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:32.827308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:32.836086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 22/44] 10.222.0.44 () {56 vars in 1322 bytes} [Wed Oct 12 12:19:32 2022] GET /identity/v3/role_assignments?user.id=cd7cad0b678b4f0182359eae55570531&scope.domain.id=default&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 530 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:32.955741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b1bb5-14ad-4c6f-af30-49cbd7f4bd4e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:32.999092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:33.057909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b1bb5-14ad-4c6f-af30-49cbd7f4bd4e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:33.092572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43abe5-ea38-45c8-9bd0-97e92f8bb50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:33.093201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43abe5-ea38-45c8-9bd0-97e92f8bb50b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.093485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43abe5-ea38-45c8-9bd0-97e92f8bb50b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.093761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43abe5-ea38-45c8-9bd0-97e92f8bb50b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.094879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43abe5-ea38-45c8-9bd0-97e92f8bb50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:33.096314 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43abe5-ea38-45c8-9bd0-97e92f8bb50b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:33.097868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 23/45] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:33.104195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:33.131524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:33.201956 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4229f93-4015-4350-aa14-bf7716c4c8ff None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:33.235138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c69c93-8521-4a99-b8df-c3ea579e7016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:33.235688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c69c93-8521-4a99-b8df-c3ea579e7016 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.235976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c69c93-8521-4a99-b8df-c3ea579e7016 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.236255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c69c93-8521-4a99-b8df-c3ea579e7016 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.236901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c69c93-8521-4a99-b8df-c3ea579e7016 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:33.238386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c69c93-8521-4a99-b8df-c3ea579e7016 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:33.340209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 23/46] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:33 2022] POST /identity/v3/projects => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:35.457884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e8b98-98bc-4055-af7d-8ce237215b1b None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:35.458406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e8b98-98bc-4055-af7d-8ce237215b1b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:35.458406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e8b98-98bc-4055-af7d-8ce237215b1b None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:35.459723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 24/47] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:35.464944 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feef785-a68e-4fe6-8af6-26d57fe9180e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:35.465337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feef785-a68e-4fe6-8af6-26d57fe9180e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:35.465503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feef785-a68e-4fe6-8af6-26d57fe9180e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:35.484045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8feef785-a68e-4fe6-8af6-26d57fe9180e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:35.485226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8feef785-a68e-4fe6-8af6-26d57fe9180e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:35.548981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 24/48] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:35.818747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e8b98-98bc-4055-af7d-8ce237215b1b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:35.918890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e8b98-98bc-4055-af7d-8ce237215b1b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:35.927692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:35.952373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26779e18-d212-4b11-81d3-64111ecb3c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:35.952992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779e18-d212-4b11-81d3-64111ecb3c96 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:35.953274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779e18-d212-4b11-81d3-64111ecb3c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:35.953550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26779e18-d212-4b11-81d3-64111ecb3c96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:35.954684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26779e18-d212-4b11-81d3-64111ecb3c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:35.956121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26779e18-d212-4b11-81d3-64111ecb3c96 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:35.957663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 25/49] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:35.963188 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feef785-a68e-4fe6-8af6-26d57fe9180e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:35.973330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:36.059420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feef785-a68e-4fe6-8af6-26d57fe9180e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:36.092449 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a65da-89d6-4397-9a5f-a4978155f498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:36.093000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a65da-89d6-4397-9a5f-a4978155f498 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:36.093281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a65da-89d6-4397-9a5f-a4978155f498 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:36.093555 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a65da-89d6-4397-9a5f-a4978155f498 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:36.094228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a65da-89d6-4397-9a5f-a4978155f498 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:36.095679 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a65da-89d6-4397-9a5f-a4978155f498 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:36.174257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 25/50] 10.222.0.44 () {60 vars in 1140 bytes} [Wed Oct 12 12:19:35 2022] POST /identity/v3/projects => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:38.299379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee2e6c-4047-4b4c-b604-475309b6d3d9 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:38.299878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee2e6c-4047-4b4c-b604-475309b6d3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:38.299982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee2e6c-4047-4b4c-b604-475309b6d3d9 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:38.301275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 26/51] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:38.306521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934c46-67b5-439c-a7dc-a99b46b03110 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:38.306940 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934c46-67b5-439c-a7dc-a99b46b03110 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:38.307080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934c46-67b5-439c-a7dc-a99b46b03110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:38.324963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5d934c46-67b5-439c-a7dc-a99b46b03110 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:38.326123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5d934c46-67b5-439c-a7dc-a99b46b03110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:38.379717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:38.390673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 26/52] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:38.657768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee2e6c-4047-4b4c-b604-475309b6d3d9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:38.756684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee2e6c-4047-4b4c-b604-475309b6d3d9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:38.789415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bd9e6-0e03-4e3a-b267-d95d2fd5313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:38.790060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd9e6-0e03-4e3a-b267-d95d2fd5313b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:38.790348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd9e6-0e03-4e3a-b267-d95d2fd5313b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:38.790623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd9e6-0e03-4e3a-b267-d95d2fd5313b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:38.791726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bd9e6-0e03-4e3a-b267-d95d2fd5313b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:38.793159 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bd9e6-0e03-4e3a-b267-d95d2fd5313b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:38.794736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 27/53] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:38.800115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d934c46-67b5-439c-a7dc-a99b46b03110 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:38.896202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d934c46-67b5-439c-a7dc-a99b46b03110 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:39.031826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d03052-0aed-4f08-a359-8966da9316e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:39.032343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03052-0aed-4f08-a359-8966da9316e6 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:39.032627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03052-0aed-4f08-a359-8966da9316e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:39.032900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03052-0aed-4f08-a359-8966da9316e6 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:39.033539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d03052-0aed-4f08-a359-8966da9316e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:39.034992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d03052-0aed-4f08-a359-8966da9316e6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:39.153313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 27/54] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:19:38 2022] POST /identity/v3/users => generated 275 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:42.402074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf77c0-e7a4-4908-95b8-4533c3432d2d None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.402581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf77c0-e7a4-4908-95b8-4533c3432d2d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.402581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf77c0-e7a4-4908-95b8-4533c3432d2d None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.403890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 28/55] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:42.409252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e7b33-59e0-4ebe-b565-fb08f973cd87 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.409652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e7b33-59e0-4ebe-b565-fb08f973cd87 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.409814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e7b33-59e0-4ebe-b565-fb08f973cd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.419787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8c8e-969e-445e-90d5-20d96ee9691b None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.421323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8c8e-969e-445e-90d5-20d96ee9691b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.421817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8c8e-969e-445e-90d5-20d96ee9691b None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.423435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 29/56] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:42.429264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9a7b-33a1-4dd9-801d-ba56caaba93c None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.429807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9a7b-33a1-4dd9-801d-ba56caaba93c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.430387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9a7b-33a1-4dd9-801d-ba56caaba93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.432002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1d1e7b33-59e0-4ebe-b565-fb08f973cd87 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:42.433457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1d1e7b33-59e0-4ebe-b565-fb08f973cd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:42.462610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-272e9a7b-33a1-4dd9-801d-ba56caaba93c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:42.463826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-272e9a7b-33a1-4dd9-801d-ba56caaba93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:42.522316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:42.554362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:42.570743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 28/57] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:42.604262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 30/58] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:42.654498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8803f-5f65-402e-ab6d-f8d455b2de78 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.655108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8803f-5f65-402e-ab6d-f8d455b2de78 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.655609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8803f-5f65-402e-ab6d-f8d455b2de78 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.657174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 29/59] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:42.662581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4e80e-f8b6-4b2e-96ee-3a11978231dc None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.663095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4e80e-f8b6-4b2e-96ee-3a11978231dc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.663569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4e80e-f8b6-4b2e-96ee-3a11978231dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.687268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-61c4e80e-f8b6-4b2e-96ee-3a11978231dc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:42.688787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-61c4e80e-f8b6-4b2e-96ee-3a11978231dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:42.784055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 31/60] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:42.823715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20e110-3efb-40e1-8169-db8313fa56cf None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.824332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20e110-3efb-40e1-8169-db8313fa56cf None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.824824 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20e110-3efb-40e1-8169-db8313fa56cf None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.826465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 30/61] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:42.831801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095b3a2-2d3e-4b5e-92bd-8d8443a25ee7 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.832096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095b3a2-2d3e-4b5e-92bd-8d8443a25ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.832380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095b3a2-2d3e-4b5e-92bd-8d8443a25ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.859167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-0095b3a2-2d3e-4b5e-92bd-8d8443a25ee7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:42.860352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0095b3a2-2d3e-4b5e-92bd-8d8443a25ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:42.928787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20e110-3efb-40e1-8169-db8313fa56cf None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:42.972582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 32/62] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:19:43.040999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20e110-3efb-40e1-8169-db8313fa56cf None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.080409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095b3a2-2d3e-4b5e-92bd-8d8443a25ee7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.083420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bca903-91e5-434d-8ab3-54ae5a52ea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.084097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bca903-91e5-434d-8ab3-54ae5a52ea2f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.084567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bca903-91e5-434d-8ab3-54ae5a52ea2f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.084853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bca903-91e5-434d-8ab3-54ae5a52ea2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.129285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bca903-91e5-434d-8ab3-54ae5a52ea2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:43.130275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-88bca903-91e5-434d-8ab3-54ae5a52ea2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:43.136009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:43.136846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:43.137589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 31/63] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:43.147436 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bca903-91e5-434d-8ab3-54ae5a52ea2f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.195582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.218078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095b3a2-2d3e-4b5e-92bd-8d8443a25ee7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.256352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a2dc-e648-4d26-b618-ef370cd2a0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.257146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a2dc-e648-4d26-b618-ef370cd2a0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.257433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a2dc-e648-4d26-b618-ef370cd2a0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.257740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a2dc-e648-4d26-b618-ef370cd2a0a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.262442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bca903-91e5-434d-8ab3-54ae5a52ea2f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.268904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0a2dc-e648-4d26-b618-ef370cd2a0a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-c2e0a2dc-e648-4d26-b618-ef370cd2a0a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:43.274741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:43.275578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:43.276333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 33/64] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:43.276941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a2dc-e648-4d26-b618-ef370cd2a0a6 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.302401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a58972-5d8f-4875-928f-8419751c889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.303111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58972-5d8f-4875-928f-8419751c889b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.303318 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58972-5d8f-4875-928f-8419751c889b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.303595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58972-5d8f-4875-928f-8419751c889b None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.304261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a58972-5d8f-4875-928f-8419751c889b None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:43.304570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a58972-5d8f-4875-928f-8419751c889b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.306017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a58972-5d8f-4875-928f-8419751c889b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.317825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 32/65] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.319048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308b7e-2a10-406c-9d3e-d96c76da8a44 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.319333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308b7e-2a10-406c-9d3e-d96c76da8a44 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.319605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308b7e-2a10-406c-9d3e-d96c76da8a44 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.321001 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 33/66] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:43.324027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82308b7e-2a10-406c-9d3e-d96c76da8a44 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.375727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.382332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a2dc-e648-4d26-b618-ef370cd2a0a6 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.416539 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e5adf-6bc6-47b7-a403-114b7de956dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.417157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e5adf-6bc6-47b7-a403-114b7de956dc None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.417469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e5adf-6bc6-47b7-a403-114b7de956dc None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.417743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e5adf-6bc6-47b7-a403-114b7de956dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.423435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.427716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e5adf-6bc6-47b7-a403-114b7de956dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:43.428166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82308b7e-2a10-406c-9d3e-d96c76da8a44 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-719e5adf-6bc6-47b7-a403-114b7de956dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:43.432557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:43.435498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 34/67] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:43.436258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e5adf-6bc6-47b7-a403-114b7de956dc None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.466142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.466688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.466973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.467249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.467906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:43.468210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.469591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.478525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 34/68] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.479320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.539417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e5adf-6bc6-47b7-a403-114b7de956dc None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.572772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698df49-1658-4377-8f2d-5743cd961276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.573396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698df49-1658-4377-8f2d-5743cd961276 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.573683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698df49-1658-4377-8f2d-5743cd961276 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.573982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698df49-1658-4377-8f2d-5743cd961276 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.575653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687441a6-e1b2-4c7e-a9b9-3324cc04d926 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.584307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698df49-1658-4377-8f2d-5743cd961276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-8698df49-1658-4377-8f2d-5743cd961276 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:43.589630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:43.590507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:43.591252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 35/69] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:43.595612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a398-cc8f-475a-8c22-7c5bf8f719a6 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.595973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a398-cc8f-475a-8c22-7c5bf8f719a6 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.596265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a398-cc8f-475a-8c22-7c5bf8f719a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.609597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff071c80-4a19-4edb-b11f-ee5424edec18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.610252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071c80-4a19-4edb-b11f-ee5424edec18 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.610543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071c80-4a19-4edb-b11f-ee5424edec18 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.610822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071c80-4a19-4edb-b11f-ee5424edec18 None admin] PATH_INFO: `/v3/users/69e6e73e380845b897bd84a9478b968e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.614723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7ee0a398-cc8f-475a-8c22-7c5bf8f719a6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:43.615959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7ee0a398-cc8f-475a-8c22-7c5bf8f719a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:43.625739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.625972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff071c80-4a19-4edb-b11f-ee5424edec18 None admin] RBAC: Authorizing `identity:get_user(user_id=69e6e73e380845b897bd84a9478b968e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.627445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff071c80-4a19-4edb-b11f-ee5424edec18 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.629070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 35/70] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/users/69e6e73e380845b897bd84a9478b968e => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.630004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff071c80-4a19-4edb-b11f-ee5424edec18 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.648840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.683995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 36/71] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:43.684834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0a398-cc8f-475a-8c22-7c5bf8f719a6 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.744170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff071c80-4a19-4edb-b11f-ee5424edec18 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.780271 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.780878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.781154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.781587 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.782288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:43.782736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.784110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.788445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0a398-cc8f-475a-8c22-7c5bf8f719a6 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.796969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 36/72] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.796969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.832045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48af5ce-4a83-405a-8246-f70f22fad98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.832795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48af5ce-4a83-405a-8246-f70f22fad98f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.833078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48af5ce-4a83-405a-8246-f70f22fad98f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.833347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48af5ce-4a83-405a-8246-f70f22fad98f None admin] PATH_INFO: `/v3/users/cd7cad0b678b4f0182359eae55570531` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.834558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48af5ce-4a83-405a-8246-f70f22fad98f None admin] RBAC: Authorizing `identity:get_user(user_id=cd7cad0b678b4f0182359eae55570531)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.835983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48af5ce-4a83-405a-8246-f70f22fad98f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.842669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 37/73] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/users/cd7cad0b678b4f0182359eae55570531 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.842669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48af5ce-4a83-405a-8246-f70f22fad98f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.857364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.885588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.906653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6c2e5-c4c2-4340-a33c-895086fad32f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.945012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48af5ce-4a83-405a-8246-f70f22fad98f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.950983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.951604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.951893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.952171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.952838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:43.953155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.954571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.963557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 37/74] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.964353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.985231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e811e0-2292-4ad3-9069-2d2cded9ea43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.985856 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e811e0-2292-4ad3-9069-2d2cded9ea43 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.986168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e811e0-2292-4ad3-9069-2d2cded9ea43 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.986441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e811e0-2292-4ad3-9069-2d2cded9ea43 None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.987566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e811e0-2292-4ad3-9069-2d2cded9ea43 None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.988998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e811e0-2292-4ad3-9069-2d2cded9ea43 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.990639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 38/75] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.991388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e811e0-2292-4ad3-9069-2d2cded9ea43 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.071570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:44.074531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74880b-b72e-4940-9d84-f07e5d6faf1b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.090281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e811e0-2292-4ad3-9069-2d2cded9ea43 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.108018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb6c8-bab4-440d-a847-e306dd02ae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.108653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb6c8-bab4-440d-a847-e306dd02ae59 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.108939 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb6c8-bab4-440d-a847-e306dd02ae59 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.109216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb6c8-bab4-440d-a847-e306dd02ae59 None admin] PATH_INFO: `/v3/users/69e6e73e380845b897bd84a9478b968e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.110387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fb6c8-bab4-440d-a847-e306dd02ae59 None admin] RBAC: Authorizing `identity:get_user(user_id=69e6e73e380845b897bd84a9478b968e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.111832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fb6c8-bab4-440d-a847-e306dd02ae59 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.113405 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 38/76] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/users/69e6e73e380845b897bd84a9478b968e => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:44.114195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb6c8-bab4-440d-a847-e306dd02ae59 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.128025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:44.128844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1b69-7dd8-4907-8e5e-5199e7694a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.129476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1b69-7dd8-4907-8e5e-5199e7694a2e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.129763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1b69-7dd8-4907-8e5e-5199e7694a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.130073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1b69-7dd8-4907-8e5e-5199e7694a2e None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.131206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec1b69-7dd8-4907-8e5e-5199e7694a2e None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.132640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec1b69-7dd8-4907-8e5e-5199e7694a2e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.134996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 39/77] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:44.135760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1b69-7dd8-4907-8e5e-5199e7694a2e None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.219336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb6c8-bab4-440d-a847-e306dd02ae59 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.232587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1b69-7dd8-4907-8e5e-5199e7694a2e None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.252407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b9ce9-48aa-4aec-b421-97a6808b2457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.253035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9ce9-48aa-4aec-b421-97a6808b2457 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.253320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9ce9-48aa-4aec-b421-97a6808b2457 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.253596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b9ce9-48aa-4aec-b421-97a6808b2457 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.254879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b9ce9-48aa-4aec-b421-97a6808b2457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.256657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b9ce9-48aa-4aec-b421-97a6808b2457 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.258245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 39/78] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:44.259517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b9ce9-48aa-4aec-b421-97a6808b2457 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.265953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2dc87-fe28-4d86-ae4d-64f8f27f0259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.266579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2dc87-fe28-4d86-ae4d-64f8f27f0259 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.266860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2dc87-fe28-4d86-ae4d-64f8f27f0259 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.267132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2dc87-fe28-4d86-ae4d-64f8f27f0259 None admin] PATH_INFO: `/v3/users/69e6e73e380845b897bd84a9478b968e` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.268264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2dc87-fe28-4d86-ae4d-64f8f27f0259 None admin] RBAC: Authorizing `identity:get_user(user_id=69e6e73e380845b897bd84a9478b968e)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.269731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2dc87-fe28-4d86-ae4d-64f8f27f0259 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.271317 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 40/79] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:44 2022] GET /identity/v3/users/69e6e73e380845b897bd84a9478b968e => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:44.272125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2dc87-fe28-4d86-ae4d-64f8f27f0259 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.289821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:44.336598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:44.357572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b9ce9-48aa-4aec-b421-97a6808b2457 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.371725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2dc87-fe28-4d86-ae4d-64f8f27f0259 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.390410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15748c24-9dc0-4d20-923b-0314d4af1239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.391069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15748c24-9dc0-4d20-923b-0314d4af1239 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.391350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15748c24-9dc0-4d20-923b-0314d4af1239 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.391627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15748c24-9dc0-4d20-923b-0314d4af1239 None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383/users/69e6e73e380845b897bd84a9478b968e/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.393537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15748c24-9dc0-4d20-923b-0314d4af1239 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc485e87ac48e7ae6a07d85c170383, user_id=69e6e73e380845b897bd84a9478b968e, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.395013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15748c24-9dc0-4d20-923b-0314d4af1239 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.404561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201bc69-cf3c-4a07-b0d5-7c3102434a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.405183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201bc69-cf3c-4a07-b0d5-7c3102434a9d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.405465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201bc69-cf3c-4a07-b0d5-7c3102434a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.405740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201bc69-cf3c-4a07-b0d5-7c3102434a9d None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.406881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201bc69-cf3c-4a07-b0d5-7c3102434a9d None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.408313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201bc69-cf3c-4a07-b0d5-7c3102434a9d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.409956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 41/80] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:44 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:44.410708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201bc69-cf3c-4a07-b0d5-7c3102434a9d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.464974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 40/81] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:44 2022] PUT /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383/users/69e6e73e380845b897bd84a9478b968e/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:44.465858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15748c24-9dc0-4d20-923b-0314d4af1239 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.496545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:44.514817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201bc69-cf3c-4a07-b0d5-7c3102434a9d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.553043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b024b4d-0537-4165-98c1-acda1f9d33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.553721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024b4d-0537-4165-98c1-acda1f9d33cb None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.554032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024b4d-0537-4165-98c1-acda1f9d33cb None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.554311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024b4d-0537-4165-98c1-acda1f9d33cb None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383/users/cd7cad0b678b4f0182359eae55570531/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.556919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b024b4d-0537-4165-98c1-acda1f9d33cb None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc485e87ac48e7ae6a07d85c170383, user_id=cd7cad0b678b4f0182359eae55570531, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.558463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b024b4d-0537-4165-98c1-acda1f9d33cb None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.589277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15748c24-9dc0-4d20-923b-0314d4af1239 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.626635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bfd31-d72e-44c4-9759-441c98df198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.627255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bfd31-d72e-44c4-9759-441c98df198d None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.627533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bfd31-d72e-44c4-9759-441c98df198d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.627810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bfd31-d72e-44c4-9759-441c98df198d None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.628471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bfd31-d72e-44c4-9759-441c98df198d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.629917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5bfd31-d72e-44c4-9759-441c98df198d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.640309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:44.672039 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 42/82] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:44 2022] PUT /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383/users/cd7cad0b678b4f0182359eae55570531/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:44.672887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b024b4d-0537-4165-98c1-acda1f9d33cb None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.785545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 41/83] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:19:44 2022] POST /identity/v3/users => generated 289 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:44.790631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bfd31-d72e-44c4-9759-441c98df198d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.802527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b024b4d-0537-4165-98c1-acda1f9d33cb None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.850139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ecf0-99f7-411d-8807-14616c5f5210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.851080 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ecf0-99f7-411d-8807-14616c5f5210 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.851518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ecf0-99f7-411d-8807-14616c5f5210 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.851921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ecf0-99f7-411d-8807-14616c5f5210 None admin] PATH_INFO: `/v3/projects/bf0147d30f6442b79bd804e404216655` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.854003 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4ecf0-99f7-411d-8807-14616c5f5210 None admin] RBAC: Authorizing `identity:get_project(project_id=bf0147d30f6442b79bd804e404216655)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.855709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4ecf0-99f7-411d-8807-14616c5f5210 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:44.857614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 43/84] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:44 2022] GET /identity/v3/projects/bf0147d30f6442b79bd804e404216655 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:44.865099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ecf0-99f7-411d-8807-14616c5f5210 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:44.887772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:44.948550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5bfd31-d72e-44c4-9759-441c98df198d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:44.988072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd5d23-aa17-47a2-a3d0-58a4cb43e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:44.988664 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd5d23-aa17-47a2-a3d0-58a4cb43e09f None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:44.989052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd5d23-aa17-47a2-a3d0-58a4cb43e09f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:44.989465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd5d23-aa17-47a2-a3d0-58a4cb43e09f None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383/users/69e6e73e380845b897bd84a9478b968e/roles/5c4536b29c2a4ad2a8d05d6e94e6a186` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:44.991802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd5d23-aa17-47a2-a3d0-58a4cb43e09f None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc485e87ac48e7ae6a07d85c170383, user_id=69e6e73e380845b897bd84a9478b968e, role_id=5c4536b29c2a4ad2a8d05d6e94e6a186)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:44.993540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd5d23-aa17-47a2-a3d0-58a4cb43e09f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:45.010356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ecf0-99f7-411d-8807-14616c5f5210 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:45.049292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058734d-4aa8-4d75-8d3c-3efb95c9bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:45.049988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058734d-4aa8-4d75-8d3c-3efb95c9bcd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:45.050267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058734d-4aa8-4d75-8d3c-3efb95c9bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:45.050550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058734d-4aa8-4d75-8d3c-3efb95c9bcd8 None admin] PATH_INFO: `/v3/projects/bf0147d30f6442b79bd804e404216655/users/69e6e73e380845b897bd84a9478b968e/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:45.052633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058734d-4aa8-4d75-8d3c-3efb95c9bcd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf0147d30f6442b79bd804e404216655, user_id=69e6e73e380845b897bd84a9478b968e, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:45.054177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058734d-4aa8-4d75-8d3c-3efb95c9bcd8 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:45.075819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 42/85] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:44 2022] PUT /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383/users/69e6e73e380845b897bd84a9478b968e/roles/5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:45.125901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 44/86] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:44 2022] PUT /identity/v3/projects/bf0147d30f6442b79bd804e404216655/users/69e6e73e380845b897bd84a9478b968e/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:47.211962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e51fa0-b529-4fdf-8730-1fe3b5ef3562 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:47.212507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e51fa0-b529-4fdf-8730-1fe3b5ef3562 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:47.212507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e51fa0-b529-4fdf-8730-1fe3b5ef3562 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:47.213873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 43/87] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:47.219142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa3151-a086-473a-bb26-fa1bf97ac48a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:47.219541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa3151-a086-473a-bb26-fa1bf97ac48a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:47.219722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa3151-a086-473a-bb26-fa1bf97ac48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:47.238771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-4caa3151-a086-473a-bb26-fa1bf97ac48a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:47.240023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4caa3151-a086-473a-bb26-fa1bf97ac48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:47.315919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 45/88] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:47.621733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b3fb6-6e8f-484e-951e-8f28e9c0d169 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:47.622115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b3fb6-6e8f-484e-951e-8f28e9c0d169 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:47.622324 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b3fb6-6e8f-484e-951e-8f28e9c0d169 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:47.623683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 44/89] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:47.629297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c631-5dd9-43ff-bb13-aa10fe1aae53 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:47.629595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c631-5dd9-43ff-bb13-aa10fe1aae53 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:47.629924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84c631-5dd9-43ff-bb13-aa10fe1aae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:47.636656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:47.651930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-3c84c631-5dd9-43ff-bb13-aa10fe1aae53 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:47.653113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3c84c631-5dd9-43ff-bb13-aa10fe1aae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:47.733653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 46/90] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:47.822557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b3fb6-6e8f-484e-951e-8f28e9c0d169 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:47.892570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:47.956910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b3fb6-6e8f-484e-951e-8f28e9c0d169 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:47.976066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311a5d3-93af-405c-a7be-0c570eb7f455 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:47.976382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311a5d3-93af-405c-a7be-0c570eb7f455 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:47.976634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311a5d3-93af-405c-a7be-0c570eb7f455 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:47.979032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 47/91] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:47.984573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c7540-e6cd-4c14-9038-b6516305b216 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:47.984862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c7540-e6cd-4c14-9038-b6516305b216 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:47.985140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c7540-e6cd-4c14-9038-b6516305b216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.012377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-f09c7540-e6cd-4c14-9038-b6516305b216 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:48.013600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f09c7540-e6cd-4c14-9038-b6516305b216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:48.019581 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9414-498f-4919-ada6-23fd193c40d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:48.020239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9414-498f-4919-ada6-23fd193c40d3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.020522 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9414-498f-4919-ada6-23fd193c40d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.020847 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9414-498f-4919-ada6-23fd193c40d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.044907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf9414-498f-4919-ada6-23fd193c40d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-0faf9414-498f-4919-ada6-23fd193c40d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:48.051577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:48.052464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:48.053670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 45/92] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:48.058848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9414-498f-4919-ada6-23fd193c40d3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:48.190958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 48/93] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:48.215670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c7540-e6cd-4c14-9038-b6516305b216 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:48.279549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9414-498f-4919-ada6-23fd193c40d3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:48.298550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:48.317612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c46c4f-e1a9-4532-8a70-38ddcc3576f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:48.318490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c46c4f-e1a9-4532-8a70-38ddcc3576f6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.318917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c46c4f-e1a9-4532-8a70-38ddcc3576f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.319335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c46c4f-e1a9-4532-8a70-38ddcc3576f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.320675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c46c4f-e1a9-4532-8a70-38ddcc3576f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:48.321676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c46c4f-e1a9-4532-8a70-38ddcc3576f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:48.323461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c46c4f-e1a9-4532-8a70-38ddcc3576f6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:48.333963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 46/94] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:19:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:48.341605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece390a0-71a9-4bad-9ea5-056c91787490 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.342213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece390a0-71a9-4bad-9ea5-056c91787490 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.342856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece390a0-71a9-4bad-9ea5-056c91787490 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.343905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:48.346208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 47/95] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:48 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:48.347495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece390a0-71a9-4bad-9ea5-056c91787490 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:48.419744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c7540-e6cd-4c14-9038-b6516305b216 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:48.481378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec193258-9144-4dc2-b027-093fa2667b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:48.482363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec193258-9144-4dc2-b027-093fa2667b11 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.482853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec193258-9144-4dc2-b027-093fa2667b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.483266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec193258-9144-4dc2-b027-093fa2667b11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.494759 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec193258-9144-4dc2-b027-093fa2667b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:48.497497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece390a0-71a9-4bad-9ea5-056c91787490 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-ec193258-9144-4dc2-b027-093fa2667b11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:48.500122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:48.500993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:48.501768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 49/96] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:48.506506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b416-5f5d-4a7d-bcd2-f9675b7e831f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.506506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b416-5f5d-4a7d-bcd2-f9675b7e831f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.506506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b416-5f5d-4a7d-bcd2-f9675b7e831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.521996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d2f4b416-5f5d-4a7d-bcd2-f9675b7e831f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:48.523250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d2f4b416-5f5d-4a7d-bcd2-f9675b7e831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:48.549597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd26e70-b5b3-4a96-b0db-a79e4d430111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:48.549913 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd26e70-b5b3-4a96-b0db-a79e4d430111 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.550205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd26e70-b5b3-4a96-b0db-a79e4d430111 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.550488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd26e70-b5b3-4a96-b0db-a79e4d430111 None admin] PATH_INFO: `/v3/users/69e6e73e380845b897bd84a9478b968e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.551812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd26e70-b5b3-4a96-b0db-a79e4d430111 None admin] RBAC: Authorizing `identity:get_user(user_id=69e6e73e380845b897bd84a9478b968e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:48.553763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd26e70-b5b3-4a96-b0db-a79e4d430111 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:48.560866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 48/97] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:48 2022] GET /identity/v3/users/69e6e73e380845b897bd84a9478b968e => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:48.560866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd26e70-b5b3-4a96-b0db-a79e4d430111 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:48.611146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 50/98] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:48.612536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4b416-5f5d-4a7d-bcd2-f9675b7e831f None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:48.630289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:48.680278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:48.693373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd26e70-b5b3-4a96-b0db-a79e4d430111 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:48.729898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4b416-5f5d-4a7d-bcd2-f9675b7e831f None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:48.730819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f5bff-7776-4580-aa2b-c24666ca57f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:48.731659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f5bff-7776-4580-aa2b-c24666ca57f1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.731894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f5bff-7776-4580-aa2b-c24666ca57f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.732285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f5bff-7776-4580-aa2b-c24666ca57f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.733486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f5bff-7776-4580-aa2b-c24666ca57f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:48.734992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f5bff-7776-4580-aa2b-c24666ca57f1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:48.736778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 49/99] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:48.737457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f5bff-7776-4580-aa2b-c24666ca57f1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:48.772074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6aa02-dd0c-4d2a-bb0e-6dfcea32371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:48.772681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6aa02-dd0c-4d2a-bb0e-6dfcea32371f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.772961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6aa02-dd0c-4d2a-bb0e-6dfcea32371f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.773232 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6aa02-dd0c-4d2a-bb0e-6dfcea32371f None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.774458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6aa02-dd0c-4d2a-bb0e-6dfcea32371f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:48.774875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6aa02-dd0c-4d2a-bb0e-6dfcea32371f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:48.776246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6aa02-dd0c-4d2a-bb0e-6dfcea32371f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:48.784589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 51/100] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:48.785620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f12c-faaf-42ac-88f1-764e2c65c9f0 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.785902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f12c-faaf-42ac-88f1-764e2c65c9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.786202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94f12c-faaf-42ac-88f1-764e2c65c9f0 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.787519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 52/101] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:48.791537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa63b8-b6f2-42b2-9cdf-840055bb4505 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.792228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa63b8-b6f2-42b2-9cdf-840055bb4505 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.793009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa63b8-b6f2-42b2-9cdf-840055bb4505 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.794457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 53/102] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:48.799313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa63b8-b6f2-42b2-9cdf-840055bb4505 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:48.850269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f5bff-7776-4580-aa2b-c24666ca57f1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:48.863579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:48.884202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1584e-baa9-4567-9629-8685d4b14cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:48.884802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1584e-baa9-4567-9629-8685d4b14cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.885080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1584e-baa9-4567-9629-8685d4b14cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.885352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1584e-baa9-4567-9629-8685d4b14cb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.913662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa63b8-b6f2-42b2-9cdf-840055bb4505 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:48.927610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:48.948495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472203e-72be-41ff-8f94-7356b3cfa84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:48.949100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472203e-72be-41ff-8f94-7356b3cfa84d None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:48.949385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472203e-72be-41ff-8f94-7356b3cfa84d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:48.949660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472203e-72be-41ff-8f94-7356b3cfa84d None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:48.950340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472203e-72be-41ff-8f94-7356b3cfa84d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:48.951779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472203e-72be-41ff-8f94-7356b3cfa84d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:48.953826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1584e-baa9-4567-9629-8685d4b14cb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:48.955313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1584e-baa9-4567-9629-8685d4b14cb2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:49.051214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 54/103] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:48 2022] POST /identity/v3/projects => generated 302 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:49.052080 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472203e-72be-41ff-8f94-7356b3cfa84d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.055467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 50/104] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:19:49.056598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c334f3-bbe0-46d4-8c56-be5ca3fe24be None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.056898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c334f3-bbe0-46d4-8c56-be5ca3fe24be None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.057172 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c334f3-bbe0-46d4-8c56-be5ca3fe24be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.084625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-07c334f3-bbe0-46d4-8c56-be5ca3fe24be None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:49.085884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-07c334f3-bbe0-46d4-8c56-be5ca3fe24be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:49.095093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:49.165987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 51/105] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:49.165987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecba42-2f4a-4485-bc4f-9bbf9f26cc66 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.165987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecba42-2f4a-4485-bc4f-9bbf9f26cc66 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.165987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecba42-2f4a-4485-bc4f-9bbf9f26cc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.186069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e0ecba42-2f4a-4485-bc4f-9bbf9f26cc66 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:49.186798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472203e-72be-41ff-8f94-7356b3cfa84d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:49.187285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e0ecba42-2f4a-4485-bc4f-9bbf9f26cc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:49.228739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57645852-3dd8-4050-9d17-9d6175676aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:49.229379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645852-3dd8-4050-9d17-9d6175676aec None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.229667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645852-3dd8-4050-9d17-9d6175676aec None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.229959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645852-3dd8-4050-9d17-9d6175676aec None admin] PATH_INFO: `/v3/users/cd7cad0b678b4f0182359eae55570531` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.231153 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57645852-3dd8-4050-9d17-9d6175676aec None admin] RBAC: Authorizing `identity:get_user(user_id=cd7cad0b678b4f0182359eae55570531)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:49.233423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57645852-3dd8-4050-9d17-9d6175676aec None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:49.235217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 55/106] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/users/cd7cad0b678b4f0182359eae55570531 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:49.240202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57645852-3dd8-4050-9d17-9d6175676aec None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.271368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 52/107] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:49.274277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecba42-2f4a-4485-bc4f-9bbf9f26cc66 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.283756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:49.360632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:49.380688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57645852-3dd8-4050-9d17-9d6175676aec None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:49.399922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecba42-2f4a-4485-bc4f-9bbf9f26cc66 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:49.426603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318b44-e7b0-4ff3-9ce1-521a2741fdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:49.427456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318b44-e7b0-4ff3-9ce1-521a2741fdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.428048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318b44-e7b0-4ff3-9ce1-521a2741fdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.428487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25318b44-e7b0-4ff3-9ce1-521a2741fdc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.441327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5403-0cd6-4f9f-81b3-8295788df8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:49.442151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5403-0cd6-4f9f-81b3-8295788df8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.442407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25318b44-e7b0-4ff3-9ce1-521a2741fdc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:49.442623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5403-0cd6-4f9f-81b3-8295788df8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.443001 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a5403-0cd6-4f9f-81b3-8295788df8c5 None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.444301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a5403-0cd6-4f9f-81b3-8295788df8c5 None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:49.446452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a5403-0cd6-4f9f-81b3-8295788df8c5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:49.452005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 53/108] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:49.452005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5403-0cd6-4f9f-81b3-8295788df8c5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-25318b44-e7b0-4ff3-9ce1-521a2741fdc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:49.452699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:49.453522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:49.454302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:49.456911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 56/109] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:49.457595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318b44-e7b0-4ff3-9ce1-521a2741fdc9 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.564579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a5403-0cd6-4f9f-81b3-8295788df8c5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:49.565197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:49.575974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25318b44-e7b0-4ff3-9ce1-521a2741fdc9 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:49.603748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a487ac-5e1d-4f3d-947e-a071def4348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:49.604621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a487ac-5e1d-4f3d-947e-a071def4348d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.604907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a487ac-5e1d-4f3d-947e-a071def4348d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.605184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a487ac-5e1d-4f3d-947e-a071def4348d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.619229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:49.623439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95256e4a-fed0-4cbe-9d9b-88461148b44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:49.624689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95256e4a-fed0-4cbe-9d9b-88461148b44f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.625384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95256e4a-fed0-4cbe-9d9b-88461148b44f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.625852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95256e4a-fed0-4cbe-9d9b-88461148b44f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.639312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95256e4a-fed0-4cbe-9d9b-88461148b44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-95256e4a-fed0-4cbe-9d9b-88461148b44f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:49.644363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:49.645301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:49.646158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 57/110] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:49.650801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95256e4a-fed0-4cbe-9d9b-88461148b44f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.687408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a487ac-5e1d-4f3d-947e-a071def4348d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:49.688917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a487ac-5e1d-4f3d-947e-a071def4348d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:49.757682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95256e4a-fed0-4cbe-9d9b-88461148b44f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:49.792528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:49.793383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.794007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.794295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.795028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] RBAC: Adding query filter params (role.id=92af8b532e69461b9088cc236a91aa39, scope.project.id=43bc485e87ac48e7ae6a07d85c170383, user.id=69e6e73e380845b897bd84a9478b968e) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:49.795397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:49.796855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:49.805973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 58/111] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/role_assignments?user.id=69e6e73e380845b897bd84a9478b968e&scope.project.id=43bc485e87ac48e7ae6a07d85c170383&role.id=92af8b532e69461b9088cc236a91aa39 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:49.805973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.806164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 54/112] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:19:49.806995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a487ac-5e1d-4f3d-947e-a071def4348d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.825653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:49.861349 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:49.918883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c334f-8764-4f9d-813c-351ebf51f2fa None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:49.947549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a487ac-5e1d-4f3d-947e-a071def4348d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:49.958144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:49.958344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.958596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.958792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.959533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:49.959839 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:49.961384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:49.971330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 59/113] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:49.972241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.994652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9a373-e171-4bc8-8e28-caf7917d9aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:49.995688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9a373-e171-4bc8-8e28-caf7917d9aac None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.996063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9a373-e171-4bc8-8e28-caf7917d9aac None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.996344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9a373-e171-4bc8-8e28-caf7917d9aac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.008149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9a373-e171-4bc8-8e28-caf7917d9aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6eb9a373-e171-4bc8-8e28-caf7917d9aac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:50.013295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:50.014250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:50.015128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 55/114] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:50.015902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9a373-e171-4bc8-8e28-caf7917d9aac None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.081103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.107988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8cf37-4565-4826-abb5-075e475557ca None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.131976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.135023 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9a373-e171-4bc8-8e28-caf7917d9aac None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.153614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898b189-b505-4236-8373-43540b36be43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.154251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b189-b505-4236-8373-43540b36be43 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.154532 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b189-b505-4236-8373-43540b36be43 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.154801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898b189-b505-4236-8373-43540b36be43 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.155457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898b189-b505-4236-8373-43540b36be43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:50.155760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898b189-b505-4236-8373-43540b36be43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.157136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898b189-b505-4236-8373-43540b36be43 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.170146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 60/115] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.170975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898b189-b505-4236-8373-43540b36be43 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.172759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b7ec5-4bb3-4751-90fe-ec85bce61bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.173401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7ec5-4bb3-4751-90fe-ec85bce61bde None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.173708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7ec5-4bb3-4751-90fe-ec85bce61bde None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.173965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7ec5-4bb3-4751-90fe-ec85bce61bde None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.175832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b7ec5-4bb3-4751-90fe-ec85bce61bde None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.177274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b7ec5-4bb3-4751-90fe-ec85bce61bde None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.183788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 56/116] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.183788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b7ec5-4bb3-4751-90fe-ec85bce61bde None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.278265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898b189-b505-4236-8373-43540b36be43 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.284546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b7ec5-4bb3-4751-90fe-ec85bce61bde None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.313552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b376-c5f6-41ee-94b7-886cfae060c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.314270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b376-c5f6-41ee-94b7-886cfae060c2 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.314609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b376-c5f6-41ee-94b7-886cfae060c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.314890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b376-c5f6-41ee-94b7-886cfae060c2 None admin] PATH_INFO: `/v3/users/69e6e73e380845b897bd84a9478b968e` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.316188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277b376-c5f6-41ee-94b7-886cfae060c2 None admin] RBAC: Authorizing `identity:get_user(user_id=69e6e73e380845b897bd84a9478b968e)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.317636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277b376-c5f6-41ee-94b7-886cfae060c2 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.319234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 61/117] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/users/69e6e73e380845b897bd84a9478b968e => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.319234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b376-c5f6-41ee-94b7-886cfae060c2 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.324477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.325110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.325398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.325677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.326378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:50.326687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.328236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.332609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.342287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 57/118] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.342287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.373026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.428588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b376-c5f6-41ee-94b7-886cfae060c2 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.444481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48f9a6-d1c5-44e5-9717-91d590389d79 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.462781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e96862-b45c-44d1-a00f-310a04e47c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.463504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e96862-b45c-44d1-a00f-310a04e47c8c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.463863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e96862-b45c-44d1-a00f-310a04e47c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.464209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e96862-b45c-44d1-a00f-310a04e47c8c None admin] PATH_INFO: `/v3/users/4a9aa60322f44b279fe3d4a9ca00aa14` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.477959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e96862-b45c-44d1-a00f-310a04e47c8c None admin] RBAC: Authorizing `identity:get_user(user_id=4a9aa60322f44b279fe3d4a9ca00aa14)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.479494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e96862-b45c-44d1-a00f-310a04e47c8c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.480807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.485760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 62/119] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/users/4a9aa60322f44b279fe3d4a9ca00aa14 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.485882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.485882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.485882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.486105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e96862-b45c-44d1-a00f-310a04e47c8c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.486197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=43bc485e87ac48e7ae6a07d85c170383, user.id=cd7cad0b678b4f0182359eae55570531) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:50.486197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.486197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.492006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 58/120] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/role_assignments?user.id=cd7cad0b678b4f0182359eae55570531&scope.project.id=43bc485e87ac48e7ae6a07d85c170383&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.492712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.578542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.618175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e96862-b45c-44d1-a00f-310a04e47c8c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.622929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.634521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033474d-fd4e-4a49-9ce3-f6c95debffac None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.657323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425b45a-cbe4-4105-8455-38060e9624ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.657965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425b45a-cbe4-4105-8455-38060e9624ad None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.658322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425b45a-cbe4-4105-8455-38060e9624ad None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.658602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425b45a-cbe4-4105-8455-38060e9624ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.683597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48863622-9b0c-4bc4-8697-f5e4aea2f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.684529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863622-9b0c-4bc4-8697-f5e4aea2f936 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.685262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863622-9b0c-4bc4-8697-f5e4aea2f936 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.686205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48863622-9b0c-4bc4-8697-f5e4aea2f936 None admin] PATH_INFO: `/v3/users/69e6e73e380845b897bd84a9478b968e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.688565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48863622-9b0c-4bc4-8697-f5e4aea2f936 None admin] RBAC: Authorizing `identity:get_user(user_id=69e6e73e380845b897bd84a9478b968e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.690344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48863622-9b0c-4bc4-8697-f5e4aea2f936 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.692568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 59/121] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/users/69e6e73e380845b897bd84a9478b968e => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.693730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48863622-9b0c-4bc4-8697-f5e4aea2f936 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.743399 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425b45a-cbe4-4105-8455-38060e9624ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.744863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425b45a-cbe4-4105-8455-38060e9624ad None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.814598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.823828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48863622-9b0c-4bc4-8697-f5e4aea2f936 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.853216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 63/122] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:19:50.853216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425b45a-cbe4-4105-8455-38060e9624ad None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.864354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05256c3-bb0f-4fea-8a09-0a175cc202ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.865094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05256c3-bb0f-4fea-8a09-0a175cc202ff None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.865621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05256c3-bb0f-4fea-8a09-0a175cc202ff None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.865991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05256c3-bb0f-4fea-8a09-0a175cc202ff None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.867289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05256c3-bb0f-4fea-8a09-0a175cc202ff None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.869608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05256c3-bb0f-4fea-8a09-0a175cc202ff None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.876753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 60/123] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.876753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05256c3-bb0f-4fea-8a09-0a175cc202ff None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.908579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.955867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425b45a-cbe4-4105-8455-38060e9624ad None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.980460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05256c3-bb0f-4fea-8a09-0a175cc202ff None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.990169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef862-5232-4a42-a7b2-46c0ef415f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.990953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef862-5232-4a42-a7b2-46c0ef415f35 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.991396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef862-5232-4a42-a7b2-46c0ef415f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.991669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef862-5232-4a42-a7b2-46c0ef415f35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.017571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27c52d-42d7-436e-bb95-d0bc2a6a5856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:51.018234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27c52d-42d7-436e-bb95-d0bc2a6a5856 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:51.018516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27c52d-42d7-436e-bb95-d0bc2a6a5856 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:51.018792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27c52d-42d7-436e-bb95-d0bc2a6a5856 None admin] PATH_INFO: `/v3/projects/bf0147d30f6442b79bd804e404216655` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.019921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27c52d-42d7-436e-bb95-d0bc2a6a5856 None admin] RBAC: Authorizing `identity:get_project(project_id=bf0147d30f6442b79bd804e404216655)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:51.021364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27c52d-42d7-436e-bb95-d0bc2a6a5856 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:51.023025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 61/124] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/projects/bf0147d30f6442b79bd804e404216655 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:51.023769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27c52d-42d7-436e-bb95-d0bc2a6a5856 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:51.034823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:51.071922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ef862-5232-4a42-a7b2-46c0ef415f35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:51.074051 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ef862-5232-4a42-a7b2-46c0ef415f35 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:51.132728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:51.137091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27c52d-42d7-436e-bb95-d0bc2a6a5856 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:51.173072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda07e87-1e92-46fa-a4b4-23c4ba079158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:51.173826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda07e87-1e92-46fa-a4b4-23c4ba079158 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:51.174202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda07e87-1e92-46fa-a4b4-23c4ba079158 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:51.174550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda07e87-1e92-46fa-a4b4-23c4ba079158 None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383/users/4a9aa60322f44b279fe3d4a9ca00aa14/roles/672667b4e6d14a17ac7cb28a93d1c48e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.177008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda07e87-1e92-46fa-a4b4-23c4ba079158 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc485e87ac48e7ae6a07d85c170383, user_id=4a9aa60322f44b279fe3d4a9ca00aa14, role_id=672667b4e6d14a17ac7cb28a93d1c48e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:51.178557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda07e87-1e92-46fa-a4b4-23c4ba079158 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:51.184319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 64/125] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:19:51.184319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef862-5232-4a42-a7b2-46c0ef415f35 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:51.247769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:51.254560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 62/126] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:51 2022] PUT /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383/users/4a9aa60322f44b279fe3d4a9ca00aa14/roles/672667b4e6d14a17ac7cb28a93d1c48e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:51.255402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda07e87-1e92-46fa-a4b4-23c4ba079158 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:51.292927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef862-5232-4a42-a7b2-46c0ef415f35 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:51.341533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267358fd-ae21-4625-bfd2-04b4cc26e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:51.342459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267358fd-ae21-4625-bfd2-04b4cc26e146 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:51.342748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267358fd-ae21-4625-bfd2-04b4cc26e146 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:51.343027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267358fd-ae21-4625-bfd2-04b4cc26e146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.343900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267358fd-ae21-4625-bfd2-04b4cc26e146 None admin] RBAC: Adding query filter params (role.id=92af8b532e69461b9088cc236a91aa39, scope.project.id=bf0147d30f6442b79bd804e404216655, user.id=69e6e73e380845b897bd84a9478b968e) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:51.344383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267358fd-ae21-4625-bfd2-04b4cc26e146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:51.346599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267358fd-ae21-4625-bfd2-04b4cc26e146 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:51.355973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 65/127] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:51 2022] GET /identity/v3/role_assignments?user.id=69e6e73e380845b897bd84a9478b968e&scope.project.id=bf0147d30f6442b79bd804e404216655&role.id=92af8b532e69461b9088cc236a91aa39 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:51.403645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723af9d-0e25-4410-8630-067c79c0d6b9 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:51.403988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723af9d-0e25-4410-8630-067c79c0d6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:51.404213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723af9d-0e25-4410-8630-067c79c0d6b9 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.405830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 66/128] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:51.406543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda07e87-1e92-46fa-a4b4-23c4ba079158 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:51.415535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bafd32-708e-4786-a60c-07c9b283af94 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:51.415882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bafd32-708e-4786-a60c-07c9b283af94 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:51.416171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bafd32-708e-4786-a60c-07c9b283af94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.436132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c2bafd32-708e-4786-a60c-07c9b283af94 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:51.437378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c2bafd32-708e-4786-a60c-07c9b283af94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:51.455510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7ce8f-3997-4822-91d9-32f61ab87384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:51.456483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7ce8f-3997-4822-91d9-32f61ab87384 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:51.456776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7ce8f-3997-4822-91d9-32f61ab87384 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:51.457059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7ce8f-3997-4822-91d9-32f61ab87384 None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.459050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7ce8f-3997-4822-91d9-32f61ab87384 None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:51.460683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7ce8f-3997-4822-91d9-32f61ab87384 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:51.464467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 63/129] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:51 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:51.470832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7ce8f-3997-4822-91d9-32f61ab87384 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:51.504287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:51.530157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 67/130] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:51.587873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7ce8f-3997-4822-91d9-32f61ab87384 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:51.624132 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:51.625018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:51.625472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:51.625917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.626836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] RBAC: Adding query filter params (role.id=5c4536b29c2a4ad2a8d05d6e94e6a186, scope.project.id=43bc485e87ac48e7ae6a07d85c170383, user.id=69e6e73e380845b897bd84a9478b968e) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:51.627372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:51.629035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:51.635497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 64/131] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:51 2022] GET /identity/v3/role_assignments?user.id=69e6e73e380845b897bd84a9478b968e&scope.project.id=43bc485e87ac48e7ae6a07d85c170383&role.id=5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:51.804631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bafd32-708e-4786-a60c-07c9b283af94 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:51.869383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:51.908298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bafd32-708e-4786-a60c-07c9b283af94 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:51.941954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0e881-aa2c-4b77-88f1-57eb711e56f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:51.942600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e881-aa2c-4b77-88f1-57eb711e56f0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:51.942882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e881-aa2c-4b77-88f1-57eb711e56f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:51.943156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e881-aa2c-4b77-88f1-57eb711e56f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:51.944248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0e881-aa2c-4b77-88f1-57eb711e56f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:51.946227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0e881-aa2c-4b77-88f1-57eb711e56f0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:51.947802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 68/132] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:51.953237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:52.043903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:52.052809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d034ee8-343b-45c0-887f-aff86018fb3f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:52.090145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf3783-3e12-43f2-bb78-894ebc6157e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:52.090761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf3783-3e12-43f2-bb78-894ebc6157e6 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:52.091044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf3783-3e12-43f2-bb78-894ebc6157e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:52.091317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf3783-3e12-43f2-bb78-894ebc6157e6 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:52.091964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf3783-3e12-43f2-bb78-894ebc6157e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:52.093404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf3783-3e12-43f2-bb78-894ebc6157e6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:52.175679 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 65/133] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:19:51 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:53.476529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74079d-2e42-4272-a1e1-f30f51459cd7 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:53.477134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74079d-2e42-4272-a1e1-f30f51459cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:53.477134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74079d-2e42-4272-a1e1-f30f51459cd7 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:53.478431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 69/134] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:53.483736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1cd8f-888b-4207-9206-acb9ccceecbd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:53.484130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1cd8f-888b-4207-9206-acb9ccceecbd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:53.484294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1cd8f-888b-4207-9206-acb9ccceecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:53.502583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f7a1cd8f-888b-4207-9206-acb9ccceecbd None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:53.503768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f7a1cd8f-888b-4207-9206-acb9ccceecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:53.571602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 66/135] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:19:53.837842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74079d-2e42-4272-a1e1-f30f51459cd7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:53.948263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74079d-2e42-4272-a1e1-f30f51459cd7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:53.982594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:53.986983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61576b3-dc57-4bb4-a30a-f9b1ff30c4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:53.987630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61576b3-dc57-4bb4-a30a-f9b1ff30c4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:53.987916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61576b3-dc57-4bb4-a30a-f9b1ff30c4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:53.988199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61576b3-dc57-4bb4-a30a-f9b1ff30c4b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:53.999668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61576b3-dc57-4bb4-a30a-f9b1ff30c4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-f61576b3-dc57-4bb4-a30a-f9b1ff30c4b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:54.004555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:54.007154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:54.007942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 70/136] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:54.014065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1cd8f-888b-4207-9206-acb9ccceecbd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:54.094894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:54.113726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1cd8f-888b-4207-9206-acb9ccceecbd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:54.148772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:54.149970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.150261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.150538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.151210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:54.151519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:54.152911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:54.161610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 67/137] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:19:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:54.167485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61576b3-dc57-4bb4-a30a-f9b1ff30c4b6 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:54.272890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61576b3-dc57-4bb4-a30a-f9b1ff30c4b6 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:54.308861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4756c52-4c0b-4b5c-816f-adec91ae3bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:54.309782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4756c52-4c0b-4b5c-816f-adec91ae3bec None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.310267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4756c52-4c0b-4b5c-816f-adec91ae3bec None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.310740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4756c52-4c0b-4b5c-816f-adec91ae3bec None admin] PATH_INFO: `/v3/users/4a9aa60322f44b279fe3d4a9ca00aa14` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.312065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4756c52-4c0b-4b5c-816f-adec91ae3bec None admin] RBAC: Authorizing `identity:get_user(user_id=4a9aa60322f44b279fe3d4a9ca00aa14)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:54.313690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4756c52-4c0b-4b5c-816f-adec91ae3bec None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:54.315800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 71/138] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:54 2022] GET /identity/v3/users/4a9aa60322f44b279fe3d4a9ca00aa14 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:54.322578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:54.435247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f6697-9789-4668-9f5a-f04d665046af None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:54.452689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31510595-e957-4dfb-b0a8-350a99a7264b None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.453324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31510595-e957-4dfb-b0a8-350a99a7264b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.453826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31510595-e957-4dfb-b0a8-350a99a7264b None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.456251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 72/139] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:54.465433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5398-432a-4bb4-b9ee-886b8846e8cc None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.466109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5398-432a-4bb4-b9ee-886b8846e8cc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.466945 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5398-432a-4bb4-b9ee-886b8846e8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.480674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb14777-59d1-40b8-bc83-0e31a8243519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:54.481304 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14777-59d1-40b8-bc83-0e31a8243519 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.481674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14777-59d1-40b8-bc83-0e31a8243519 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.481983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb14777-59d1-40b8-bc83-0e31a8243519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.493246 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:54.499703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-0ebd5398-432a-4bb4-b9ee-886b8846e8cc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:54.502228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0ebd5398-432a-4bb4-b9ee-886b8846e8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:54.562870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb14777-59d1-40b8-bc83-0e31a8243519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:54.564965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb14777-59d1-40b8-bc83-0e31a8243519 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:54.576075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:54.590319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 73/140] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:54.591813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980fcc7-e49b-4aaa-b324-58d893896a34 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.592102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980fcc7-e49b-4aaa-b324-58d893896a34 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.592376 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980fcc7-e49b-4aaa-b324-58d893896a34 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.593792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 74/141] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:54.599951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49669a3e-b6bb-448b-bcba-7a898e0257ba None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.600389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49669a3e-b6bb-448b-bcba-7a898e0257ba None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.601086 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49669a3e-b6bb-448b-bcba-7a898e0257ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.627163 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-49669a3e-b6bb-448b-bcba-7a898e0257ba None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:54.629109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-49669a3e-b6bb-448b-bcba-7a898e0257ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:54.682944 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 68/142] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:19:54.684260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b466ffa-506a-437b-bac5-a63ba2a344da None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.684551 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b466ffa-506a-437b-bac5-a63ba2a344da None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.684826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b466ffa-506a-437b-bac5-a63ba2a344da None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.686291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 69/143] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:54.688818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b466ffa-506a-437b-bac5-a63ba2a344da None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:54.713171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 75/144] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:54.714835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8937-4cc0-4fd1-99f4-86ecd3362d3e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.715188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8937-4cc0-4fd1-99f4-86ecd3362d3e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.715535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c8937-4cc0-4fd1-99f4-86ecd3362d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.740462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:54.744709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-173c8937-4cc0-4fd1-99f4-86ecd3362d3e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:54.746093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-173c8937-4cc0-4fd1-99f4-86ecd3362d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:54.812332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b466ffa-506a-437b-bac5-a63ba2a344da None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:54.840216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 76/145] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:54.855374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4b7d0-ab9b-47bd-8e57-2aa3af77df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:54.856187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b7d0-ab9b-47bd-8e57-2aa3af77df4d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:54.856614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b7d0-ab9b-47bd-8e57-2aa3af77df4d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:54.857082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b7d0-ab9b-47bd-8e57-2aa3af77df4d None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:54.858370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4b7d0-ab9b-47bd-8e57-2aa3af77df4d None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:54.860051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4b7d0-ab9b-47bd-8e57-2aa3af77df4d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:54.861878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 70/146] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:54 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:54.869092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c8937-4cc0-4fd1-99f4-86ecd3362d3e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:54.870155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4b7d0-ab9b-47bd-8e57-2aa3af77df4d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:54.925773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:54.974992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4b7d0-ab9b-47bd-8e57-2aa3af77df4d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:54.988340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c8937-4cc0-4fd1-99f4-86ecd3362d3e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:54.990920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.012736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee502713-d315-4739-b1c0-427dbaa7ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.013688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee502713-d315-4739-b1c0-427dbaa7ad02 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.014000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee502713-d315-4739-b1c0-427dbaa7ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.014298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee502713-d315-4739-b1c0-427dbaa7ad02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.025120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee502713-d315-4739-b1c0-427dbaa7ad02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:55.025289 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.026100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.026540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.027046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.027912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] RBAC: Adding query filter params (role.id=672667b4e6d14a17ac7cb28a93d1c48e, scope.project.id=43bc485e87ac48e7ae6a07d85c170383, user.id=4a9aa60322f44b279fe3d4a9ca00aa14) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:55.028367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.030077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-ee502713-d315-4739-b1c0-427dbaa7ad02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:55.030205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:55.031398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 71/147] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:55.037049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee502713-d315-4739-b1c0-427dbaa7ad02 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.041548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 77/148] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:54 2022] GET /identity/v3/role_assignments?user.id=4a9aa60322f44b279fe3d4a9ca00aa14&scope.project.id=43bc485e87ac48e7ae6a07d85c170383&role.id=672667b4e6d14a17ac7cb28a93d1c48e => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.042249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.156229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee502713-d315-4739-b1c0-427dbaa7ad02 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.177668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caac486-29dd-46d0-837c-4d47f2ec1d71 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.192882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.194293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd692c3-6875-46c8-b5b0-ca55c1cbd260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.195276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd692c3-6875-46c8-b5b0-ca55c1cbd260 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.195562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd692c3-6875-46c8-b5b0-ca55c1cbd260 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.195868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd692c3-6875-46c8-b5b0-ca55c1cbd260 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.208456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd692c3-6875-46c8-b5b0-ca55c1cbd260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-2cd692c3-6875-46c8-b5b0-ca55c1cbd260 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:55.213875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:55.214777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 72/149] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:55.215983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd692c3-6875-46c8-b5b0-ca55c1cbd260 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.217658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.218315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.218595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.219261 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.219544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:55.219849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.221341 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.231583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 78/150] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.232648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.254696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.331363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd692c3-6875-46c8-b5b0-ca55c1cbd260 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.336047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c2a96-3a79-4d95-9cda-0762a9f23971 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.365816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f75db-4572-425b-91db-ba0026970384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.366468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f75db-4572-425b-91db-ba0026970384 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.366747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f75db-4572-425b-91db-ba0026970384 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.367017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f75db-4572-425b-91db-ba0026970384 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.368136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f75db-4572-425b-91db-ba0026970384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.369559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f75db-4572-425b-91db-ba0026970384 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.369752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.370366 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.370642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.370910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.371133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 73/151] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.371565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:55.371866 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.373212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.374951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f75db-4572-425b-91db-ba0026970384 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.381443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 79/152] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.382233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.417205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.468965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.476091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f75db-4572-425b-91db-ba0026970384 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.481391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cf50d-3bad-424e-9464-ba5d281a7462 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.509562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca4ff8-0c33-46f9-8395-84acb804719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.510155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca4ff8-0c33-46f9-8395-84acb804719d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.510429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca4ff8-0c33-46f9-8395-84acb804719d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.510695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca4ff8-0c33-46f9-8395-84acb804719d None admin] PATH_INFO: `/v3/users/6d9c87dac6484754b5ac29f818b50748` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.514715 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfa1aa-6b20-41d9-8600-99f63c01cbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.515381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfa1aa-6b20-41d9-8600-99f63c01cbef None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.515726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfa1aa-6b20-41d9-8600-99f63c01cbef None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.516082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfa1aa-6b20-41d9-8600-99f63c01cbef None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.516795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfa1aa-6b20-41d9-8600-99f63c01cbef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.518243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfa1aa-6b20-41d9-8600-99f63c01cbef None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.523021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca4ff8-0c33-46f9-8395-84acb804719d None admin] RBAC: Authorizing `identity:get_user(user_id=6d9c87dac6484754b5ac29f818b50748)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.524987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca4ff8-0c33-46f9-8395-84acb804719d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.526607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 74/153] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/users/6d9c87dac6484754b5ac29f818b50748 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.526607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca4ff8-0c33-46f9-8395-84acb804719d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.605524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 80/154] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:19:55 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:55.606381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfa1aa-6b20-41d9-8600-99f63c01cbef None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.636477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca4ff8-0c33-46f9-8395-84acb804719d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.677638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e7ba0-bc16-4302-a0ba-a5bb9204a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.678598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e7ba0-bc16-4302-a0ba-a5bb9204a01e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.678886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e7ba0-bc16-4302-a0ba-a5bb9204a01e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.679156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e7ba0-bc16-4302-a0ba-a5bb9204a01e None admin] PATH_INFO: `/v3/users/6d9c87dac6484754b5ac29f818b50748` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.680382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e7ba0-bc16-4302-a0ba-a5bb9204a01e None admin] RBAC: Authorizing `identity:get_user(user_id=6d9c87dac6484754b5ac29f818b50748)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.681855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e7ba0-bc16-4302-a0ba-a5bb9204a01e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.683546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 75/155] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/users/6d9c87dac6484754b5ac29f818b50748 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.689659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e7ba0-bc16-4302-a0ba-a5bb9204a01e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.713297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.714239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.723561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfa1aa-6b20-41d9-8600-99f63c01cbef None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.762229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0120e-2a02-47f3-9b88-5f28a6349c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.762956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0120e-2a02-47f3-9b88-5f28a6349c15 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.763422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0120e-2a02-47f3-9b88-5f28a6349c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.763589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0120e-2a02-47f3-9b88-5f28a6349c15 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.764789 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0120e-2a02-47f3-9b88-5f28a6349c15 None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.766296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0120e-2a02-47f3-9b88-5f28a6349c15 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.767973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 81/156] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.774594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0120e-2a02-47f3-9b88-5f28a6349c15 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.809517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e7ba0-bc16-4302-a0ba-a5bb9204a01e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.853848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e43c51-9e81-4ae3-b0e2-6ddf5ab214f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.854517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e43c51-9e81-4ae3-b0e2-6ddf5ab214f3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.854798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e43c51-9e81-4ae3-b0e2-6ddf5ab214f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.855069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e43c51-9e81-4ae3-b0e2-6ddf5ab214f3 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.856202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e43c51-9e81-4ae3-b0e2-6ddf5ab214f3 None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.857653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e43c51-9e81-4ae3-b0e2-6ddf5ab214f3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.859400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 76/157] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.865276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e43c51-9e81-4ae3-b0e2-6ddf5ab214f3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.896982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0120e-2a02-47f3-9b88-5f28a6349c15 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.944503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d7e18-7e79-433e-8ced-db8988ccae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.945171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d7e18-7e79-433e-8ced-db8988ccae66 None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.945445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d7e18-7e79-433e-8ced-db8988ccae66 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.946264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d7e18-7e79-433e-8ced-db8988ccae66 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae/users/6d9c87dac6484754b5ac29f818b50748/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.948185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d7e18-7e79-433e-8ced-db8988ccae66 None admin] RBAC: Authorizing `identity:create_grant(project_id=8b46ee23713a44848ffd7947384566ae, user_id=6d9c87dac6484754b5ac29f818b50748, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.949746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d7e18-7e79-433e-8ced-db8988ccae66 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.968423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.980433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e43c51-9e81-4ae3-b0e2-6ddf5ab214f3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:56.015678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f158ce-b248-46b5-b04b-b70defed07ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:56.016319 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f158ce-b248-46b5-b04b-b70defed07ed None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:56.016604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f158ce-b248-46b5-b04b-b70defed07ed None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:56.016875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f158ce-b248-46b5-b04b-b70defed07ed None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae/users/6d9c87dac6484754b5ac29f818b50748/roles/5c4536b29c2a4ad2a8d05d6e94e6a186` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:56.019077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f158ce-b248-46b5-b04b-b70defed07ed None admin] RBAC: Authorizing `identity:create_grant(project_id=8b46ee23713a44848ffd7947384566ae, user_id=6d9c87dac6484754b5ac29f818b50748, role_id=5c4536b29c2a4ad2a8d05d6e94e6a186)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:56.020574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f158ce-b248-46b5-b04b-b70defed07ed None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:56.064457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 82/158] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:55 2022] PUT /identity/v3/projects/8b46ee23713a44848ffd7947384566ae/users/6d9c87dac6484754b5ac29f818b50748/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:56.121047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 77/159] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:55 2022] PUT /identity/v3/projects/8b46ee23713a44848ffd7947384566ae/users/6d9c87dac6484754b5ac29f818b50748/roles/5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:19:57.881873 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4632764-71f7-48e0-9f91-03a79477f5e8 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:57.882463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4632764-71f7-48e0-9f91-03a79477f5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:57.882463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4632764-71f7-48e0-9f91-03a79477f5e8 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:57.884597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 83/160] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:57.890011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10abb63-6d7e-47f1-a3ca-c791862166bc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:57.890441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10abb63-6d7e-47f1-a3ca-c791862166bc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:57.890631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10abb63-6d7e-47f1-a3ca-c791862166bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:57.910598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f10abb63-6d7e-47f1-a3ca-c791862166bc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:57.911784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f10abb63-6d7e-47f1-a3ca-c791862166bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:57.947055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7649-8544-417c-9614-b27faa2b6981 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:57.947384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7649-8544-417c-9614-b27faa2b6981 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:57.947605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7649-8544-417c-9614-b27faa2b6981 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:57.949102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 84/161] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:57.953889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a114bee-cbfb-4f12-a87c-822d942bb220 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:57.954282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a114bee-cbfb-4f12-a87c-822d942bb220 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:57.954561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a114bee-cbfb-4f12-a87c-822d942bb220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:57.978405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-4a114bee-cbfb-4f12-a87c-822d942bb220 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:57.979883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4a114bee-cbfb-4f12-a87c-822d942bb220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:57.990056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 78/162] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:19:57.994607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:58.048847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 85/163] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:19:58.254436 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10abb63-6d7e-47f1-a3ca-c791862166bc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:58.287777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51e163-4d62-4554-9cc2-5e27cae2a4a2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.288062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51e163-4d62-4554-9cc2-5e27cae2a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.288339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51e163-4d62-4554-9cc2-5e27cae2a4a2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.289908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 86/164] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:58.295106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e317d-1b2f-4385-8c54-2ae15fa65096 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.295408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e317d-1b2f-4385-8c54-2ae15fa65096 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.295688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e317d-1b2f-4385-8c54-2ae15fa65096 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.296949 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 87/165] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:58.297974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a707-c5c5-420c-a6c1-015e2b2bee49 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.298249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a707-c5c5-420c-a6c1-015e2b2bee49 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.298516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4a707-c5c5-420c-a6c1-015e2b2bee49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.318015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7ba4a707-c5c5-420c-a6c1-015e2b2bee49 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:58.319458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7ba4a707-c5c5-420c-a6c1-015e2b2bee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:58.341291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:58.362157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10abb63-6d7e-47f1-a3ca-c791862166bc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:58.387883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 88/166] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:58.389199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b171-7bfc-4f73-b379-1455d9351ce2 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.389482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b171-7bfc-4f73-b379-1455d9351ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.389814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b171-7bfc-4f73-b379-1455d9351ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.397051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4532d-986f-4731-9ce1-e728d7997b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:58.397689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4532d-986f-4731-9ce1-e728d7997b1e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.397985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4532d-986f-4731-9ce1-e728d7997b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.398287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4532d-986f-4731-9ce1-e728d7997b1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.408427 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4532d-986f-4731-9ce1-e728d7997b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:58.410911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2419b171-7bfc-4f73-b379-1455d9351ce2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:58.412232 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2419b171-7bfc-4f73-b379-1455d9351ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e8b4532d-986f-4731-9ce1-e728d7997b1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:58.413263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:58.414411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 79/167] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:58.420006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4532d-986f-4731-9ce1-e728d7997b1e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:58.496259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 89/168] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:58.498030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419b171-7bfc-4f73-b379-1455d9351ce2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:58.514880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:58.524657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4532d-986f-4731-9ce1-e728d7997b1e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:58.560315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8813a-4bb2-4683-97f5-af78e0cfd7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:58.560972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8813a-4bb2-4683-97f5-af78e0cfd7df None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.561303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8813a-4bb2-4683-97f5-af78e0cfd7df None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.561629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8813a-4bb2-4683-97f5-af78e0cfd7df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.562820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8813a-4bb2-4683-97f5-af78e0cfd7df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:58.564278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8813a-4bb2-4683-97f5-af78e0cfd7df None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:58.565846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 80/169] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:58.571927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8813a-4bb2-4683-97f5-af78e0cfd7df None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:58.581661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:58.631814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419b171-7bfc-4f73-b379-1455d9351ce2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:58.673829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:58.674567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.674745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.675017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.675680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] RBAC: Adding query filter params (name=member) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:58.675984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:58.677350 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:58.686168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8813a-4bb2-4683-97f5-af78e0cfd7df None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:58.686718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 90/170] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:19:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:58.687355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:58.725860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaca32f-b69c-47f2-93e5-bda778a97725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:58.727491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaca32f-b69c-47f2-93e5-bda778a97725 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.727491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaca32f-b69c-47f2-93e5-bda778a97725 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.727669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaca32f-b69c-47f2-93e5-bda778a97725 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.728179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaca32f-b69c-47f2-93e5-bda778a97725 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:58.729597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaca32f-b69c-47f2-93e5-bda778a97725 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:58.763208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:58.791695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26222d2-f7ad-4b13-b1d2-e3c96b692954 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:58.823708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ddefa-5670-498a-8819-6cc10d5ef05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:58.824329 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ddefa-5670-498a-8819-6cc10d5ef05c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.824605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ddefa-5670-498a-8819-6cc10d5ef05c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.824871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ddefa-5670-498a-8819-6cc10d5ef05c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.833650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 81/171] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:19:58 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:58.833650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaca32f-b69c-47f2-93e5-bda778a97725 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:58.841005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ddefa-5670-498a-8819-6cc10d5ef05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-9c1ddefa-5670-498a-8819-6cc10d5ef05c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:58.848422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:58.849420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 91/172] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:58.850875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ddefa-5670-498a-8819-6cc10d5ef05c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:58.910324 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:58.953373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaca32f-b69c-47f2-93e5-bda778a97725 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:58.970820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ddefa-5670-498a-8819-6cc10d5ef05c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:58.991224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38455f-403c-4d02-abfb-ff0659ee7409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:58.992161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38455f-403c-4d02-abfb-ff0659ee7409 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.993142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38455f-403c-4d02-abfb-ff0659ee7409 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.993567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38455f-403c-4d02-abfb-ff0659ee7409 None admin] PATH_INFO: `/v3/users/bf2e86c37b5944558d346a6fba6cd0fe` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.005195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:59.009242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa38455f-403c-4d02-abfb-ff0659ee7409 None admin] RBAC: Authorizing `identity:get_user(user_id=bf2e86c37b5944558d346a6fba6cd0fe)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.010753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa38455f-403c-4d02-abfb-ff0659ee7409 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.013216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 82/173] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:58 2022] GET /identity/v3/users/bf2e86c37b5944558d346a6fba6cd0fe => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:59.013216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38455f-403c-4d02-abfb-ff0659ee7409 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.016741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1162f3-8f01-43e2-942d-566052784a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.016741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1162f3-8f01-43e2-942d-566052784a87 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.016741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1162f3-8f01-43e2-942d-566052784a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.016741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1162f3-8f01-43e2-942d-566052784a87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.030010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1162f3-8f01-43e2-942d-566052784a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-1d1162f3-8f01-43e2-942d-566052784a87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:59.036340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:59.037397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 92/174] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:59.038452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1162f3-8f01-43e2-942d-566052784a87 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.145483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38455f-403c-4d02-abfb-ff0659ee7409 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.162615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1162f3-8f01-43e2-942d-566052784a87 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.172900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:59.195296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.195950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.196255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.196526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.197423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:59.197534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.199032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.207512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75d484-b46a-437f-97c3-b7ec3b610ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.207975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 83/175] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:59.208557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75d484-b46a-437f-97c3-b7ec3b610ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.208978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75d484-b46a-437f-97c3-b7ec3b610ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.209439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75d484-b46a-437f-97c3-b7ec3b610ab3 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.210811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75d484-b46a-437f-97c3-b7ec3b610ab3 None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.210991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.213263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75d484-b46a-437f-97c3-b7ec3b610ab3 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.215375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 93/176] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:59 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:59.216404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75d484-b46a-437f-97c3-b7ec3b610ab3 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.285606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:59.337029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7e5db-445d-4d3d-8bb5-6d6bfa22d729 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.342432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75d484-b46a-437f-97c3-b7ec3b610ab3 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.374978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.375845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.376225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.376600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.377469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:59.377928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.379519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.387935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109c3ab-f519-41bd-b5f3-881dd58b7c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.388570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c3ab-f519-41bd-b5f3-881dd58b7c47 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.388896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c3ab-f519-41bd-b5f3-881dd58b7c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.389168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c3ab-f519-41bd-b5f3-881dd58b7c47 None admin] PATH_INFO: `/v3/users/6d9c87dac6484754b5ac29f818b50748` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.389993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 84/177] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:59.390326 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109c3ab-f519-41bd-b5f3-881dd58b7c47 None admin] RBAC: Authorizing `identity:get_user(user_id=6d9c87dac6484754b5ac29f818b50748)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.390986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.394437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109c3ab-f519-41bd-b5f3-881dd58b7c47 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.396090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 94/178] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:59 2022] GET /identity/v3/users/6d9c87dac6484754b5ac29f818b50748 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:59.396848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109c3ab-f519-41bd-b5f3-881dd58b7c47 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.448929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:59.509893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109c3ab-f519-41bd-b5f3-881dd58b7c47 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.513506 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946e519-9bbe-4443-87b4-95b9e6749b33 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.538997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:59.547499 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a9c61-408e-4e43-bb45-e9f83ace5476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.548076 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a9c61-408e-4e43-bb45-e9f83ace5476 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.548356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a9c61-408e-4e43-bb45-e9f83ace5476 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.548629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a9c61-408e-4e43-bb45-e9f83ace5476 None admin] PATH_INFO: `/v3/users/6d9c87dac6484754b5ac29f818b50748` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.549861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a9c61-408e-4e43-bb45-e9f83ace5476 None admin] RBAC: Authorizing `identity:get_user(user_id=6d9c87dac6484754b5ac29f818b50748)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.551455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a9c61-408e-4e43-bb45-e9f83ace5476 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.553132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 95/179] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:59 2022] GET /identity/v3/users/6d9c87dac6484754b5ac29f818b50748 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:59.557705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a9c61-408e-4e43-bb45-e9f83ace5476 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.561776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec20ee-ad8c-48a9-8dfb-fbd650ec42bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.562711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec20ee-ad8c-48a9-8dfb-fbd650ec42bd None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.563184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec20ee-ad8c-48a9-8dfb-fbd650ec42bd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.563648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec20ee-ad8c-48a9-8dfb-fbd650ec42bd None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae/users/bf2e86c37b5944558d346a6fba6cd0fe/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.565808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec20ee-ad8c-48a9-8dfb-fbd650ec42bd None admin] RBAC: Authorizing `identity:create_grant(project_id=8b46ee23713a44848ffd7947384566ae, user_id=bf2e86c37b5944558d346a6fba6cd0fe, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.567975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec20ee-ad8c-48a9-8dfb-fbd650ec42bd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.635199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 85/180] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:59 2022] PUT /identity/v3/projects/8b46ee23713a44848ffd7947384566ae/users/bf2e86c37b5944558d346a6fba6cd0fe/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:59.636344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec20ee-ad8c-48a9-8dfb-fbd650ec42bd None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.686304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a9c61-408e-4e43-bb45-e9f83ace5476 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.765421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a2c97-be77-4318-9909-fd4710665022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.766035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a2c97-be77-4318-9909-fd4710665022 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.766372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a2c97-be77-4318-9909-fd4710665022 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.766652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a2c97-be77-4318-9909-fd4710665022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.768721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec20ee-ad8c-48a9-8dfb-fbd650ec42bd None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.772206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:59.811817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c98a7-02de-4bf0-a2e3-35cce2f957d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.813322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c98a7-02de-4bf0-a2e3-35cce2f957d4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.813804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c98a7-02de-4bf0-a2e3-35cce2f957d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.814360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c98a7-02de-4bf0-a2e3-35cce2f957d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.904808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c98a7-02de-4bf0-a2e3-35cce2f957d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.907280 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c98a7-02de-4bf0-a2e3-35cce2f957d4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.936295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:59.950641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a2c97-be77-4318-9909-fd4710665022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.952217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a2c97-be77-4318-9909-fd4710665022 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:00.044089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 86/181] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:00.050465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c98a7-02de-4bf0-a2e3-35cce2f957d4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:00.054223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:00.073638 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 96/182] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:00.079072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a2c97-be77-4318-9909-fd4710665022 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:00.165834 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c98a7-02de-4bf0-a2e3-35cce2f957d4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:00.204761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f0701-b02f-4e22-b17d-d05d16922e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:00.206031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f0701-b02f-4e22-b17d-d05d16922e12 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:00.206528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f0701-b02f-4e22-b17d-d05d16922e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:00.206991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f0701-b02f-4e22-b17d-d05d16922e12 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:00.208339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f0701-b02f-4e22-b17d-d05d16922e12 None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:00.210083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f0701-b02f-4e22-b17d-d05d16922e12 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:00.212265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 87/183] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:00 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:00.218114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f0701-b02f-4e22-b17d-d05d16922e12 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:00.250478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:00.258182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a2c97-be77-4318-9909-fd4710665022 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:00.309351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:00.319577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651642f-3d06-44a5-a1c1-c2346c814b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:00.327924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651642f-3d06-44a5-a1c1-c2346c814b06 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:00.327924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651642f-3d06-44a5-a1c1-c2346c814b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:00.327924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651642f-3d06-44a5-a1c1-c2346c814b06 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:00.327924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651642f-3d06-44a5-a1c1-c2346c814b06 None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:00.327924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651642f-3d06-44a5-a1c1-c2346c814b06 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:00.330100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 97/184] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:00 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:00.333395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651642f-3d06-44a5-a1c1-c2346c814b06 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:00.369135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f0701-b02f-4e22-b17d-d05d16922e12 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:00.445193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbd79e-8439-4b6b-a8b6-3523e99e6581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:00.446819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd79e-8439-4b6b-a8b6-3523e99e6581 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:00.447400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd79e-8439-4b6b-a8b6-3523e99e6581 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:00.447868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd79e-8439-4b6b-a8b6-3523e99e6581 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:00.448790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbd79e-8439-4b6b-a8b6-3523e99e6581 None admin] RBAC: Adding query filter params (role.id=5c4536b29c2a4ad2a8d05d6e94e6a186, scope.project.id=8b46ee23713a44848ffd7947384566ae, user.id=6d9c87dac6484754b5ac29f818b50748) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:00.449290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbd79e-8439-4b6b-a8b6-3523e99e6581 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:00.450949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbd79e-8439-4b6b-a8b6-3523e99e6581 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:00.457650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 88/185] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:00 2022] GET /identity/v3/role_assignments?user.id=6d9c87dac6484754b5ac29f818b50748&scope.project.id=8b46ee23713a44848ffd7947384566ae&role.id=5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:00.502297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651642f-3d06-44a5-a1c1-c2346c814b06 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:00.559612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67e9e7-b41d-4d40-811a-8ca398f1875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:00.560238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e9e7-b41d-4d40-811a-8ca398f1875e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:00.560516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e9e7-b41d-4d40-811a-8ca398f1875e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:00.560790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e9e7-b41d-4d40-811a-8ca398f1875e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:00.561532 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67e9e7-b41d-4d40-811a-8ca398f1875e None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=8b46ee23713a44848ffd7947384566ae, user.id=6d9c87dac6484754b5ac29f818b50748) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:00.561845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67e9e7-b41d-4d40-811a-8ca398f1875e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:00.563383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67e9e7-b41d-4d40-811a-8ca398f1875e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:00.570085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 98/186] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:00 2022] GET /identity/v3/role_assignments?user.id=6d9c87dac6484754b5ac29f818b50748&scope.project.id=8b46ee23713a44848ffd7947384566ae&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:01.236850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71599-9170-4d6c-ad65-78ebfd59e537 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.237377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71599-9170-4d6c-ad65-78ebfd59e537 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.237377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71599-9170-4d6c-ad65-78ebfd59e537 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.238912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 89/187] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:01.244309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0f01-c9e2-402e-8641-bee47910d4d5 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.244702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0f01-c9e2-402e-8641-bee47910d4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.244860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc0f01-c9e2-402e-8641-bee47910d4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.256563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aef17-4216-4ab3-a70e-4a9a935c2b9d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.256842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aef17-4216-4ab3-a70e-4a9a935c2b9d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.257109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870aef17-4216-4ab3-a70e-4a9a935c2b9d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.259234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 90/188] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:20:01.263563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-afcc0f01-c9e2-402e-8641-bee47910d4d5 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:01.264035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422c3f-f03b-4f45-9ea5-bd91b29c329e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.264316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422c3f-f03b-4f45-9ea5-bd91b29c329e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.264589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f422c3f-f03b-4f45-9ea5-bd91b29c329e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.264745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-afcc0f01-c9e2-402e-8641-bee47910d4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:01.282092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4f422c3f-f03b-4f45-9ea5-bd91b29c329e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:01.283259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4f422c3f-f03b-4f45-9ea5-bd91b29c329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:01.287941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:01.333259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 99/189] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:01.352340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 91/190] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:01.604576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc0f01-c9e2-402e-8641-bee47910d4d5 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:01.614883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f422c3f-f03b-4f45-9ea5-bd91b29c329e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:01.684122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:01.709857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc0f01-c9e2-402e-8641-bee47910d4d5 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:01.720981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f422c3f-f03b-4f45-9ea5-bd91b29c329e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:01.745178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15484e-5735-4b33-a3c2-32758a14d703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:01.745819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15484e-5735-4b33-a3c2-32758a14d703 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.746132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15484e-5735-4b33-a3c2-32758a14d703 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.746406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15484e-5735-4b33-a3c2-32758a14d703 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.756617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a15484e-5735-4b33-a3c2-32758a14d703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:01.757327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13364f-a188-4069-acbc-7c7a555c3f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:01.758694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13364f-a188-4069-acbc-7c7a555c3f22 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.759162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13364f-a188-4069-acbc-7c7a555c3f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.759602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13364f-a188-4069-acbc-7c7a555c3f22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.760854 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13364f-a188-4069-acbc-7c7a555c3f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-7a15484e-5735-4b33-a3c2-32758a14d703 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:01.761968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:01.762837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:01.763623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:01.764273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13364f-a188-4069-acbc-7c7a555c3f22 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:01.764273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 92/191] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:01.766224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 100/192] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:01.768833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13364f-a188-4069-acbc-7c7a555c3f22 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:01.769644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15484e-5735-4b33-a3c2-32758a14d703 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:01.789863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:01.874152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13364f-a188-4069-acbc-7c7a555c3f22 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:01.880813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15484e-5735-4b33-a3c2-32758a14d703 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:01.908811 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:01.909416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.909694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.909991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.910654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:01.910955 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:01.912315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:01.913644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:01.917160 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd113c53-a600-472a-b16c-4d60c16b6a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:01.917771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd113c53-a600-472a-b16c-4d60c16b6a4b None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.918367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd113c53-a600-472a-b16c-4d60c16b6a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.918367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd113c53-a600-472a-b16c-4d60c16b6a4b None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.919002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd113c53-a600-472a-b16c-4d60c16b6a4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:01.920445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd113c53-a600-472a-b16c-4d60c16b6a4b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:01.921445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 93/193] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:01.926851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:01.991037 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 101/194] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:01 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:20:02.032761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80583b73-b22c-49a6-9816-da73c52512b2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:02.077611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe14510-2a93-480b-869f-8e0dcbd8709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:02.078698 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe14510-2a93-480b-869f-8e0dcbd8709d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.079182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe14510-2a93-480b-869f-8e0dcbd8709d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.079702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe14510-2a93-480b-869f-8e0dcbd8709d None admin] PATH_INFO: `/v3/users/0e5b0e821dd14197ab0a5991477b32e5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.096887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe14510-2a93-480b-869f-8e0dcbd8709d None admin] RBAC: Authorizing `identity:get_user(user_id=0e5b0e821dd14197ab0a5991477b32e5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:02.099011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe14510-2a93-480b-869f-8e0dcbd8709d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:02.100978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 94/195] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:01 2022] GET /identity/v3/users/0e5b0e821dd14197ab0a5991477b32e5 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:02.107253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd113c53-a600-472a-b16c-4d60c16b6a4b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:02.196838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:02.222260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd113c53-a600-472a-b16c-4d60c16b6a4b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:02.262882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfbecb-ce4d-4fec-a110-0d381fe88ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:02.263501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfbecb-ce4d-4fec-a110-0d381fe88ace None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.263798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfbecb-ce4d-4fec-a110-0d381fe88ace None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.264130 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfbecb-ce4d-4fec-a110-0d381fe88ace None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.265735 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfbecb-ce4d-4fec-a110-0d381fe88ace None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:02.267275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfbecb-ce4d-4fec-a110-0d381fe88ace None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:02.269077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 102/196] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:02 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:02.275230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe14510-2a93-480b-869f-8e0dcbd8709d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:02.282141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b4dbe-2d50-4d4b-b51c-a5b044c85ad6 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.282466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b4dbe-2d50-4d4b-b51c-a5b044c85ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.282701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b4dbe-2d50-4d4b-b51c-a5b044c85ad6 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.284182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 103/197] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:02.289964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f3fec-8de8-4a5f-9a8d-d898b1b7ef36 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.290252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f3fec-8de8-4a5f-9a8d-d898b1b7ef36 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.290526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f3fec-8de8-4a5f-9a8d-d898b1b7ef36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.310867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d02f3fec-8de8-4a5f-9a8d-d898b1b7ef36 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:02.312381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d02f3fec-8de8-4a5f-9a8d-d898b1b7ef36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:02.344440 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:02.391806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 104/198] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:02.402253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe14510-2a93-480b-869f-8e0dcbd8709d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:02.444964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a9b31-69a5-48f5-8544-cca6663335ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:02.446476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a9b31-69a5-48f5-8544-cca6663335ea None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.446754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a9b31-69a5-48f5-8544-cca6663335ea None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.447080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a9b31-69a5-48f5-8544-cca6663335ea None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae/users/0e5b0e821dd14197ab0a5991477b32e5/roles/672667b4e6d14a17ac7cb28a93d1c48e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.449017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a9b31-69a5-48f5-8544-cca6663335ea None admin] RBAC: Authorizing `identity:create_grant(project_id=8b46ee23713a44848ffd7947384566ae, user_id=0e5b0e821dd14197ab0a5991477b32e5, role_id=672667b4e6d14a17ac7cb28a93d1c48e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:02.450546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a9b31-69a5-48f5-8544-cca6663335ea None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:02.515057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 95/199] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:02 2022] PUT /identity/v3/projects/8b46ee23713a44848ffd7947384566ae/users/0e5b0e821dd14197ab0a5991477b32e5/roles/672667b4e6d14a17ac7cb28a93d1c48e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:02.661595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f3fec-8de8-4a5f-9a8d-d898b1b7ef36 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:02.745203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:02.776866 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f3fec-8de8-4a5f-9a8d-d898b1b7ef36 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:02.812695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433fe3-9680-41a9-907c-18fdc4eea704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:02.813335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433fe3-9680-41a9-907c-18fdc4eea704 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.813619 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433fe3-9680-41a9-907c-18fdc4eea704 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.813895 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433fe3-9680-41a9-907c-18fdc4eea704 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.824512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44433fe3-9680-41a9-907c-18fdc4eea704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-44433fe3-9680-41a9-907c-18fdc4eea704 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:02.829373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:02.830259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:02.831447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 105/200] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:02.839081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a9b31-69a5-48f5-8544-cca6663335ea None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:02.945183 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a9b31-69a5-48f5-8544-cca6663335ea None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:02.960311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:02.982458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:02.983417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.983888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.984352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.985265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:02.985759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:02.987334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:02.996184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 96/201] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:03.005964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433fe3-9680-41a9-907c-18fdc4eea704 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:03.114107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433fe3-9680-41a9-907c-18fdc4eea704 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:03.150303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:03.152646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b8c28-78c9-4d65-b059-b27745990b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:03.153283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8c28-78c9-4d65-b059-b27745990b26 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:03.153563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8c28-78c9-4d65-b059-b27745990b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:03.153838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b8c28-78c9-4d65-b059-b27745990b26 None admin] PATH_INFO: `/v3/users/bf2e86c37b5944558d346a6fba6cd0fe` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:03.154997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b8c28-78c9-4d65-b059-b27745990b26 None admin] RBAC: Authorizing `identity:get_user(user_id=bf2e86c37b5944558d346a6fba6cd0fe)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:03.156433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b8c28-78c9-4d65-b059-b27745990b26 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:03.158190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 106/202] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:03 2022] GET /identity/v3/users/bf2e86c37b5944558d346a6fba6cd0fe => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:03.163696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:03.267081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da784c-ecfc-452d-8724-3f1eab08cab7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:03.305701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04977a-0ecb-4622-8e62-a366dcdc95b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:03.307090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04977a-0ecb-4622-8e62-a366dcdc95b8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:03.307581 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04977a-0ecb-4622-8e62-a366dcdc95b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:03.308139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04977a-0ecb-4622-8e62-a366dcdc95b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:03.355093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:03.377967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04977a-0ecb-4622-8e62-a366dcdc95b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:03.379434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04977a-0ecb-4622-8e62-a366dcdc95b8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:03.483781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 97/203] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:03.489513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b8c28-78c9-4d65-b059-b27745990b26 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:03.592398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b8c28-78c9-4d65-b059-b27745990b26 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:03.630386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090015b7-43bc-4682-8b64-138d948954b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:03.631002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090015b7-43bc-4682-8b64-138d948954b0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:03.631280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090015b7-43bc-4682-8b64-138d948954b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:03.631551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090015b7-43bc-4682-8b64-138d948954b0 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:03.632665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090015b7-43bc-4682-8b64-138d948954b0 None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:03.634753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090015b7-43bc-4682-8b64-138d948954b0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:03.636407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 107/204] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:03 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:03.643036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04977a-0ecb-4622-8e62-a366dcdc95b8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:03.722451 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:03.746252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04977a-0ecb-4622-8e62-a366dcdc95b8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:03.781155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03151a-c96d-4f79-ba19-a5e28b081f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:03.781801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03151a-c96d-4f79-ba19-a5e28b081f5b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:03.782113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03151a-c96d-4f79-ba19-a5e28b081f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:03.782387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03151a-c96d-4f79-ba19-a5e28b081f5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:03.783109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03151a-c96d-4f79-ba19-a5e28b081f5b None admin] RBAC: Adding query filter params (role.id=92af8b532e69461b9088cc236a91aa39, scope.project.id=8b46ee23713a44848ffd7947384566ae, user.id=bf2e86c37b5944558d346a6fba6cd0fe) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:03.783415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03151a-c96d-4f79-ba19-a5e28b081f5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:03.784842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03151a-c96d-4f79-ba19-a5e28b081f5b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:03.790892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 98/205] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:03 2022] GET /identity/v3/role_assignments?user.id=bf2e86c37b5944558d346a6fba6cd0fe&scope.project.id=8b46ee23713a44848ffd7947384566ae&role.id=92af8b532e69461b9088cc236a91aa39 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:04.409380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992ee2-39da-48db-a23c-379fdbf9e7c6 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.409566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992ee2-39da-48db-a23c-379fdbf9e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.409841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992ee2-39da-48db-a23c-379fdbf9e7c6 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.411366 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 108/206] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:04.417288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c97093-c3f5-4588-8932-58fb1af1d1ea None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.417734 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c97093-c3f5-4588-8932-58fb1af1d1ea None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.417807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c97093-c3f5-4588-8932-58fb1af1d1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.436722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a2c97093-c3f5-4588-8932-58fb1af1d1ea None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:04.437158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9dec-87c3-4c01-acf7-5a6c88a32bd9 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.437442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9dec-87c3-4c01-acf7-5a6c88a32bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.437714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9dec-87c3-4c01-acf7-5a6c88a32bd9 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.437958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a2c97093-c3f5-4588-8932-58fb1af1d1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:04.439080 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 109/207] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:04.444030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fd053-4cdb-4b1f-9434-0cbb8cab0ea8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.444311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fd053-4cdb-4b1f-9434-0cbb8cab0ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.444744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fd053-4cdb-4b1f-9434-0cbb8cab0ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.464820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-645fd053-4cdb-4b1f-9434-0cbb8cab0ea8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:04.466029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-645fd053-4cdb-4b1f-9434-0cbb8cab0ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:04.507979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 99/208] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:04.529991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:04.535433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 110/209] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:04.737877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da9e9-190e-4b44-a03d-916e3e2be197 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.738137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da9e9-190e-4b44-a03d-916e3e2be197 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.738612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30da9e9-190e-4b44-a03d-916e3e2be197 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.740064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 100/210] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:04.745300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f240-daff-4358-bd36-0c837555d387 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.745594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f240-daff-4358-bd36-0c837555d387 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.745871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f240-daff-4358-bd36-0c837555d387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.764436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2ef1f240-daff-4358-bd36-0c837555d387 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:04.765644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2ef1f240-daff-4358-bd36-0c837555d387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:04.777630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30da9e9-190e-4b44-a03d-916e3e2be197 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:04.833607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 111/211] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:04.834484 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1f240-daff-4358-bd36-0c837555d387 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:04.851822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:04.882648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30da9e9-190e-4b44-a03d-916e3e2be197 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:04.923027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fef510-6f2c-4c64-9420-bc217f86a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:04.923769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fef510-6f2c-4c64-9420-bc217f86a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.924053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fef510-6f2c-4c64-9420-bc217f86a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.924686 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fef510-6f2c-4c64-9420-bc217f86a4f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.925528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fef510-6f2c-4c64-9420-bc217f86a4f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:04.927148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fef510-6f2c-4c64-9420-bc217f86a4f3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:04.929011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 101/212] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:04.934052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fef510-6f2c-4c64-9420-bc217f86a4f3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:04.948855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1f240-daff-4358-bd36-0c837555d387 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:04.978956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:04.987516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eac88-22bc-4c8b-9fc3-d998a05465e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:04.991169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eac88-22bc-4c8b-9fc3-d998a05465e7 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.991169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eac88-22bc-4c8b-9fc3-d998a05465e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.991169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eac88-22bc-4c8b-9fc3-d998a05465e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.000706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eac88-22bc-4c8b-9fc3-d998a05465e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-c69eac88-22bc-4c8b-9fc3-d998a05465e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:05.005948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:05.006891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:05.007717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:05.009972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 112/213] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:05.015544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eac88-22bc-4c8b-9fc3-d998a05465e7 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.035069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fef510-6f2c-4c64-9420-bc217f86a4f3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.081974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e1419-7fbb-46f1-9f7e-da0dacacf963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.082846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e1419-7fbb-46f1-9f7e-da0dacacf963 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.083346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e1419-7fbb-46f1-9f7e-da0dacacf963 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.083813 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e1419-7fbb-46f1-9f7e-da0dacacf963 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.084783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e1419-7fbb-46f1-9f7e-da0dacacf963 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.086918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e1419-7fbb-46f1-9f7e-da0dacacf963 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.099903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:05.118804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eac88-22bc-4c8b-9fc3-d998a05465e7 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.151959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.152564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.152848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.153122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.153791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] RBAC: Adding query filter params (name=member) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:05.154126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.155501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.163352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 113/214] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:05.164154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.210382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 102/215] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:04 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:20:05.210382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e1419-7fbb-46f1-9f7e-da0dacacf963 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.228523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:05.285293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906a839-8b29-453a-b24c-b3d59d0f407b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.322476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e1419-7fbb-46f1-9f7e-da0dacacf963 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.323931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8feabb-637c-498e-9c5b-0b1dcadca2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.324739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8feabb-637c-498e-9c5b-0b1dcadca2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.325026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8feabb-637c-498e-9c5b-0b1dcadca2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.325294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8feabb-637c-498e-9c5b-0b1dcadca2c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.338262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8feabb-637c-498e-9c5b-0b1dcadca2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-4c8feabb-637c-498e-9c5b-0b1dcadca2c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:05.343779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:05.344723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:05.345527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 114/216] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:05.351276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8feabb-637c-498e-9c5b-0b1dcadca2c7 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.364181 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd95d9-eef4-425b-aba7-1d58ee43a0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.364918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd95d9-eef4-425b-aba7-1d58ee43a0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.365195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd95d9-eef4-425b-aba7-1d58ee43a0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.365469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd95d9-eef4-425b-aba7-1d58ee43a0b4 None admin] PATH_INFO: `/v3/users/5a8b44cd796e4e1d8971b1b732f35f69` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.382174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd95d9-eef4-425b-aba7-1d58ee43a0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=5a8b44cd796e4e1d8971b1b732f35f69)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.383666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd95d9-eef4-425b-aba7-1d58ee43a0b4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.385490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 103/217] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/users/5a8b44cd796e4e1d8971b1b732f35f69 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:05.391284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd95d9-eef4-425b-aba7-1d58ee43a0b4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.455581 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:05.472818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8feabb-637c-498e-9c5b-0b1dcadca2c7 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.487375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:05.513536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd95d9-eef4-425b-aba7-1d58ee43a0b4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.518043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.518670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.518958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.519234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.519909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:05.520218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.521729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.534205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 115/218] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:05.542710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.563493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.564443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.564723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.564997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.565728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] RBAC: Adding query filter params (role.id=92af8b532e69461b9088cc236a91aa39, user.id=5a8b44cd796e4e1d8971b1b732f35f69, scope.system=all) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:05.566084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.568006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.577571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 104/219] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/role_assignments?user.id=5a8b44cd796e4e1d8971b1b732f35f69&scope.system=all&role.id=92af8b532e69461b9088cc236a91aa39 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:05.660815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd8614-a3f5-4a1e-936c-4fed2ac5c1e8 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.705538 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822264b5-b355-426b-9870-2399d93de3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.706301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822264b5-b355-426b-9870-2399d93de3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.706582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822264b5-b355-426b-9870-2399d93de3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.706854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822264b5-b355-426b-9870-2399d93de3f6 None admin] PATH_INFO: `/v3/users/0e5b0e821dd14197ab0a5991477b32e5` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.708111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822264b5-b355-426b-9870-2399d93de3f6 None admin] RBAC: Authorizing `identity:get_user(user_id=0e5b0e821dd14197ab0a5991477b32e5)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.709604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822264b5-b355-426b-9870-2399d93de3f6 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.711380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 116/220] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/users/0e5b0e821dd14197ab0a5991477b32e5 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:05.716965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.823364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6b8af-fb51-4153-b941-20e13a184f0e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.841433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:05.860040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae73e19-514a-4a26-8969-5424a5c9bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.860657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae73e19-514a-4a26-8969-5424a5c9bb9c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.860963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae73e19-514a-4a26-8969-5424a5c9bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.861238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae73e19-514a-4a26-8969-5424a5c9bb9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.932441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae73e19-514a-4a26-8969-5424a5c9bb9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.933897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae73e19-514a-4a26-8969-5424a5c9bb9c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:06.034441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 105/221] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:06.043010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822264b5-b355-426b-9870-2399d93de3f6 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:06.098731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:06.154828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822264b5-b355-426b-9870-2399d93de3f6 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:06.189507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c152c3b-bfa0-4aa5-953a-26a7f74d1794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:06.190255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c152c3b-bfa0-4aa5-953a-26a7f74d1794 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:06.190539 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c152c3b-bfa0-4aa5-953a-26a7f74d1794 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:06.190812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c152c3b-bfa0-4aa5-953a-26a7f74d1794 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:06.191947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c152c3b-bfa0-4aa5-953a-26a7f74d1794 None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:06.193397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c152c3b-bfa0-4aa5-953a-26a7f74d1794 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:06.195381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 117/222] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:06 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:06.201375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae73e19-514a-4a26-8969-5424a5c9bb9c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:06.222895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:06.307410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae73e19-514a-4a26-8969-5424a5c9bb9c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:06.343537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b72958-7b8a-499f-9752-980e72cd09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:06.344176 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72958-7b8a-499f-9752-980e72cd09b5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:06.344454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72958-7b8a-499f-9752-980e72cd09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:06.344722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72958-7b8a-499f-9752-980e72cd09b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:06.345470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b72958-7b8a-499f-9752-980e72cd09b5 None admin] RBAC: Adding query filter params (role.id=672667b4e6d14a17ac7cb28a93d1c48e, scope.project.id=8b46ee23713a44848ffd7947384566ae, user.id=0e5b0e821dd14197ab0a5991477b32e5) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:06.345776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b72958-7b8a-499f-9752-980e72cd09b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:06.347294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b72958-7b8a-499f-9752-980e72cd09b5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:06.354759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 106/223] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:06 2022] GET /identity/v3/role_assignments?user.id=0e5b0e821dd14197ab0a5991477b32e5&scope.project.id=8b46ee23713a44848ffd7947384566ae&role.id=672667b4e6d14a17ac7cb28a93d1c48e => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:07.542918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96095cb9-ea10-410b-9479-3a73c89c9dab None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:07.543414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96095cb9-ea10-410b-9479-3a73c89c9dab None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:07.543488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96095cb9-ea10-410b-9479-3a73c89c9dab None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:07.544803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 118/224] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:07.550015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb649e36-cfac-4545-b33f-df8af570599d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:07.550394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb649e36-cfac-4545-b33f-df8af570599d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:07.550565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb649e36-cfac-4545-b33f-df8af570599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:07.569317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fb649e36-cfac-4545-b33f-df8af570599d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:07.570583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fb649e36-cfac-4545-b33f-df8af570599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:07.621351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661793d-ad43-4a6c-9cc8-50d2c8e3cfcd None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:07.621633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661793d-ad43-4a6c-9cc8-50d2c8e3cfcd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:07.621909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661793d-ad43-4a6c-9cc8-50d2c8e3cfcd None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:07.623428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 119/225] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:07.628029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa04012-1144-457a-b01c-6a8a5d380b86 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:07.628340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa04012-1144-457a-b01c-6a8a5d380b86 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:07.628586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa04012-1144-457a-b01c-6a8a5d380b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:07.639783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 107/226] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:07.647164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-4fa04012-1144-457a-b01c-6a8a5d380b86 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:07.648333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4fa04012-1144-457a-b01c-6a8a5d380b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:07.720534 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 120/227] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:07.796571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee9ec5-e368-4527-be51-0dca3d6ef8a6 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:07.796845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee9ec5-e368-4527-be51-0dca3d6ef8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:07.797060 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee9ec5-e368-4527-be51-0dca3d6ef8a6 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:07.798624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 108/228] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:07.803434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844780b-7c41-476b-b741-77b989fe4aa0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:07.803727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844780b-7c41-476b-b741-77b989fe4aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:07.804014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844780b-7c41-476b-b741-77b989fe4aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:07.822485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a844780b-7c41-476b-b741-77b989fe4aa0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:07.823666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a844780b-7c41-476b-b741-77b989fe4aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:07.854041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:07.891065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 121/229] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:07.906532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee9ec5-e368-4527-be51-0dca3d6ef8a6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:07.918764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:07.986838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844780b-7c41-476b-b741-77b989fe4aa0 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.014031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee9ec5-e368-4527-be51-0dca3d6ef8a6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.052125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cae18c-2a85-4107-a760-ee306d6114a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.052679 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cae18c-2a85-4107-a760-ee306d6114a1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.052956 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cae18c-2a85-4107-a760-ee306d6114a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.053228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cae18c-2a85-4107-a760-ee306d6114a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.064960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cae18c-2a85-4107-a760-ee306d6114a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-10cae18c-2a85-4107-a760-ee306d6114a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:08.070505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:08.071859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:08.072814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 109/230] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:08.077757 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cae18c-2a85-4107-a760-ee306d6114a1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.109395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844780b-7c41-476b-b741-77b989fe4aa0 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.145481 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985e8c1-e938-4f49-a78c-0fee4ff70f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.146572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985e8c1-e938-4f49-a78c-0fee4ff70f45 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.146875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985e8c1-e938-4f49-a78c-0fee4ff70f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.147143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985e8c1-e938-4f49-a78c-0fee4ff70f45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.148222 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985e8c1-e938-4f49-a78c-0fee4ff70f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.149225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.149673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985e8c1-e938-4f49-a78c-0fee4ff70f45 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.151430 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 122/231] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:08.156781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985e8c1-e938-4f49-a78c-0fee4ff70f45 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.179177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cae18c-2a85-4107-a760-ee306d6114a1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.191519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.215468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.216078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.216363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.216636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.217298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:08.217602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.219028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.227296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 110/232] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:08.227910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.256700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985e8c1-e938-4f49-a78c-0fee4ff70f45 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.289603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11766f2c-1078-49ae-9859-d7ead752971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.290244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11766f2c-1078-49ae-9859-d7ead752971e None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.290524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11766f2c-1078-49ae-9859-d7ead752971e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.290793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11766f2c-1078-49ae-9859-d7ead752971e None admin] PATH_INFO: `/v3/groups` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.291408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11766f2c-1078-49ae-9859-d7ead752971e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.292823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11766f2c-1078-49ae-9859-d7ead752971e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.324828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b48db-5183-4136-a5c2-ed2ba6b24d94 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.347303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 123/233] 10.222.0.44 () {60 vars in 1134 bytes} [Wed Oct 12 12:20:08 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:20:08.348373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11766f2c-1078-49ae-9859-d7ead752971e None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.353589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.358759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061edb8-1d54-4e70-8d38-da25c4b2293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.359423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061edb8-1d54-4e70-8d38-da25c4b2293b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.359702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061edb8-1d54-4e70-8d38-da25c4b2293b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.359974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061edb8-1d54-4e70-8d38-da25c4b2293b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.372951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061edb8-1d54-4e70-8d38-da25c4b2293b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-b061edb8-1d54-4e70-8d38-da25c4b2293b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:08.378067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:08.379196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:08.380523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 111/234] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:08.385359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061edb8-1d54-4e70-8d38-da25c4b2293b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.501960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061edb8-1d54-4e70-8d38-da25c4b2293b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.517453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.524917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11766f2c-1078-49ae-9859-d7ead752971e None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.542940 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.543575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.543831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.544213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.545477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:08.545477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.546770 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.556207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 112/235] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:08.561557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.565485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18121aa-e55f-4fd1-b609-abd14931c0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.566159 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18121aa-e55f-4fd1-b609-abd14931c0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.566445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18121aa-e55f-4fd1-b609-abd14931c0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.566725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18121aa-e55f-4fd1-b609-abd14931c0e3 None admin] PATH_INFO: `/v3/users/6f354288fac64a8c81a360c3b220805f` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.580688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18121aa-e55f-4fd1-b609-abd14931c0e3 None admin] RBAC: Authorizing `identity:get_user(user_id=6f354288fac64a8c81a360c3b220805f)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.582118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18121aa-e55f-4fd1-b609-abd14931c0e3 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.583965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 124/236] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/users/6f354288fac64a8c81a360c3b220805f => generated 293 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:08.589573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18121aa-e55f-4fd1-b609-abd14931c0e3 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.620749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.671241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c0ae6-e87c-43e1-af07-eb09f8824a7d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.694898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18121aa-e55f-4fd1-b609-abd14931c0e3 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.706832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74e0ca-52ef-465c-b662-f7ae2f91bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.707368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74e0ca-52ef-465c-b662-f7ae2f91bd6b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.707649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74e0ca-52ef-465c-b662-f7ae2f91bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.707921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74e0ca-52ef-465c-b662-f7ae2f91bd6b None admin] PATH_INFO: `/v3/users/5a8b44cd796e4e1d8971b1b732f35f69` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.709068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74e0ca-52ef-465c-b662-f7ae2f91bd6b None admin] RBAC: Authorizing `identity:get_user(user_id=5a8b44cd796e4e1d8971b1b732f35f69)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.710713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74e0ca-52ef-465c-b662-f7ae2f91bd6b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.712719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 113/237] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/users/5a8b44cd796e4e1d8971b1b732f35f69 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:08.718345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74e0ca-52ef-465c-b662-f7ae2f91bd6b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.732081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff99e4-d7ab-4964-a022-e3d7a0427ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.732689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff99e4-d7ab-4964-a022-e3d7a0427ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.732968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff99e4-d7ab-4964-a022-e3d7a0427ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.733237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff99e4-d7ab-4964-a022-e3d7a0427ca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.733985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff99e4-d7ab-4964-a022-e3d7a0427ca5 None admin] RBAC: Adding query filter params (role.id=672667b4e6d14a17ac7cb28a93d1c48e, user.id=6f354288fac64a8c81a360c3b220805f, scope.system=all) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:08.734297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff99e4-d7ab-4964-a022-e3d7a0427ca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.735729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff99e4-d7ab-4964-a022-e3d7a0427ca5 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.741134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 125/238] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/role_assignments?user.id=6f354288fac64a8c81a360c3b220805f&scope.system=all&role.id=672667b4e6d14a17ac7cb28a93d1c48e => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:08.828775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74e0ca-52ef-465c-b662-f7ae2f91bd6b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.854383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.869396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af63b54-ca26-4ab1-857d-a05650384f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.870756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af63b54-ca26-4ab1-857d-a05650384f15 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.871263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af63b54-ca26-4ab1-857d-a05650384f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.871731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af63b54-ca26-4ab1-857d-a05650384f15 None admin] PATH_INFO: `/v3/system/users/5a8b44cd796e4e1d8971b1b732f35f69/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.873744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af63b54-ca26-4ab1-857d-a05650384f15 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5a8b44cd796e4e1d8971b1b732f35f69, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.875384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af63b54-ca26-4ab1-857d-a05650384f15 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.963362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 114/239] 10.222.0.44 () {58 vars in 1334 bytes} [Wed Oct 12 12:20:08 2022] PUT /identity/v3/system/users/5a8b44cd796e4e1d8971b1b732f35f69/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:20:10.535162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694260b-5653-4e7d-a070-362c313eee36 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:10.535681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694260b-5653-4e7d-a070-362c313eee36 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:10.535681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694260b-5653-4e7d-a070-362c313eee36 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:10.537066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 126/240] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:10.542294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8af36-8e92-4d32-bf38-662db16c96c9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:10.542687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8af36-8e92-4d32-bf38-662db16c96c9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:10.542863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8af36-8e92-4d32-bf38-662db16c96c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:10.561614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c9c8af36-8e92-4d32-bf38-662db16c96c9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:10.562826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c9c8af36-8e92-4d32-bf38-662db16c96c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:10.631010 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 115/241] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:10.872512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a641f-6dfe-4891-a56d-7b146e81e34c None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:10.872765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a641f-6dfe-4891-a56d-7b146e81e34c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:10.873035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a641f-6dfe-4891-a56d-7b146e81e34c None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:10.874518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 127/242] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:10.879577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2af5-5236-4d22-a1d9-5ba3eb3c380f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:10.879870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2af5-5236-4d22-a1d9-5ba3eb3c380f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:10.880150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2af5-5236-4d22-a1d9-5ba3eb3c380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:10.897278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a641f-6dfe-4891-a56d-7b146e81e34c None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:10.899564 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-433b2af5-5236-4d22-a1d9-5ba3eb3c380f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:10.900750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-433b2af5-5236-4d22-a1d9-5ba3eb3c380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:10.919165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:10.974417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 116/243] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:11.012113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a641f-6dfe-4891-a56d-7b146e81e34c None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.047277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce56742-eb1a-4618-ae44-d9b2e678121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.048108 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56742-eb1a-4618-ae44-d9b2e678121b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.048529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56742-eb1a-4618-ae44-d9b2e678121b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.048967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56742-eb1a-4618-ae44-d9b2e678121b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.050231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce56742-eb1a-4618-ae44-d9b2e678121b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:11.051816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce56742-eb1a-4618-ae44-d9b2e678121b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:11.053535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 128/244] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:11.059113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2af5-5236-4d22-a1d9-5ba3eb3c380f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.080341 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06339-6722-4581-b64e-eae396109dbd None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.080625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06339-6722-4581-b64e-eae396109dbd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.080897 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06339-6722-4581-b64e-eae396109dbd None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.082443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 129/245] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:20:11.087522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733ae9d-8319-46b2-b5b8-7f82e1eae771 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.087796 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733ae9d-8319-46b2-b5b8-7f82e1eae771 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.088075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733ae9d-8319-46b2-b5b8-7f82e1eae771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.107427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7733ae9d-8319-46b2-b5b8-7f82e1eae771 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:11.108621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7733ae9d-8319-46b2-b5b8-7f82e1eae771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:11.110713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:11.168121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2af5-5236-4d22-a1d9-5ba3eb3c380f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.173768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 130/246] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:11.202820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810678e-30c4-4a6d-901e-39342b9e10c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.203671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810678e-30c4-4a6d-901e-39342b9e10c1 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.204093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810678e-30c4-4a6d-901e-39342b9e10c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.204556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810678e-30c4-4a6d-901e-39342b9e10c1 None admin] PATH_INFO: `/v3/groups` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.205335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810678e-30c4-4a6d-901e-39342b9e10c1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:11.206947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810678e-30c4-4a6d-901e-39342b9e10c1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:11.246228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733ae9d-8319-46b2-b5b8-7f82e1eae771 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.263623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:11.309959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 117/247] 10.222.0.44 () {60 vars in 1134 bytes} [Wed Oct 12 12:20:11 2022] POST /identity/v3/groups => generated 226 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:20:11.359980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733ae9d-8319-46b2-b5b8-7f82e1eae771 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.399157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a63e0-a19a-4f29-befa-f4de4000f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.399802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a63e0-a19a-4f29-befa-f4de4000f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.400083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a63e0-a19a-4f29-befa-f4de4000f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.400352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a63e0-a19a-4f29-befa-f4de4000f3b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.412006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a63e0-a19a-4f29-befa-f4de4000f3b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-cf6a63e0-a19a-4f29-befa-f4de4000f3b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:11.417033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:11.418072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:11.418928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 131/248] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:11.428157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810678e-30c4-4a6d-901e-39342b9e10c1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.439238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a63e0-a19a-4f29-befa-f4de4000f3b2 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.529519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:11.549463 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810678e-30c4-4a6d-901e-39342b9e10c1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.552988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a63e0-a19a-4f29-befa-f4de4000f3b2 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.557075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:11.588669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.589281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.589559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.589960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.590740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:11.591419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:11.595047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:11.598823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e2ee0-f003-4d7a-9ab4-929b36f61481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.605412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e2ee0-f003-4d7a-9ab4-929b36f61481 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.605412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e2ee0-f003-4d7a-9ab4-929b36f61481 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.605412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e2ee0-f003-4d7a-9ab4-929b36f61481 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.608127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 118/249] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:11.612837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.615452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e2ee0-f003-4d7a-9ab4-929b36f61481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-817e2ee0-f003-4d7a-9ab4-929b36f61481 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:11.621600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:11.622560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:11.623339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:11.624887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 132/250] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:11.629291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e2ee0-f003-4d7a-9ab4-929b36f61481 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.748483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e2ee0-f003-4d7a-9ab4-929b36f61481 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.752448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160a804-7f3f-4c47-a775-6552aaf2e3a4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.797735 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.798548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.798656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.798881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.799551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] RBAC: Adding query filter params (name=member) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:11.799855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:11.801291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:11.802418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c087-dc88-400e-ada9-71a3f69971b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.803513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c087-dc88-400e-ada9-71a3f69971b2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.803941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c087-dc88-400e-ada9-71a3f69971b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.804796 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:11.805207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c087-dc88-400e-ada9-71a3f69971b2 None admin] PATH_INFO: `/v3/users/6f354288fac64a8c81a360c3b220805f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.806561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1c087-dc88-400e-ada9-71a3f69971b2 None admin] RBAC: Authorizing `identity:get_user(user_id=6f354288fac64a8c81a360c3b220805f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:11.808983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1c087-dc88-400e-ada9-71a3f69971b2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:11.813882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 133/252] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:11.814048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 119/252] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:11 2022] GET /identity/v3/users/6f354288fac64a8c81a360c3b220805f => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:11.817619 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.821980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c087-dc88-400e-ada9-71a3f69971b2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.866715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:11.942421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd0767-6c3a-4843-a212-428d6f72cb61 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.946016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c087-dc88-400e-ada9-71a3f69971b2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.983610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bbdea-b30a-4726-b3f6-b065cbd5dc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.983841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6495c0-2675-4be9-afc3-d30ff437114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.984302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6495c0-2675-4be9-afc3-d30ff437114e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.984581 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6495c0-2675-4be9-afc3-d30ff437114e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.984856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6495c0-2675-4be9-afc3-d30ff437114e None admin] PATH_INFO: `/v3/users/5a8b44cd796e4e1d8971b1b732f35f69` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.985498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bbdea-b30a-4726-b3f6-b065cbd5dc46 None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.986572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6495c0-2675-4be9-afc3-d30ff437114e None admin] RBAC: Authorizing `identity:get_user(user_id=5a8b44cd796e4e1d8971b1b732f35f69)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:11.986824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bbdea-b30a-4726-b3f6-b065cbd5dc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.987295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bbdea-b30a-4726-b3f6-b065cbd5dc46 None admin] PATH_INFO: `/v3/system/users/6f354288fac64a8c81a360c3b220805f/roles/672667b4e6d14a17ac7cb28a93d1c48e` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.987521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6495c0-2675-4be9-afc3-d30ff437114e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:11.989263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bbdea-b30a-4726-b3f6-b065cbd5dc46 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6f354288fac64a8c81a360c3b220805f, role_id=672667b4e6d14a17ac7cb28a93d1c48e)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:11.989526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 120/253] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:11 2022] GET /identity/v3/users/5a8b44cd796e4e1d8971b1b732f35f69 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:11.991194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bbdea-b30a-4726-b3f6-b065cbd5dc46 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:11.994892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6495c0-2675-4be9-afc3-d30ff437114e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:12.057487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 134/254] 10.222.0.44 () {58 vars in 1334 bytes} [Wed Oct 12 12:20:11 2022] PUT /identity/v3/system/users/6f354288fac64a8c81a360c3b220805f/roles/672667b4e6d14a17ac7cb28a93d1c48e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:12.113342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6495c0-2675-4be9-afc3-d30ff437114e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:12.116197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:12.162874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532a3bd-e416-4699-bed8-71a144239c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:12.163760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a3bd-e416-4699-bed8-71a144239c2c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:12.164278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a3bd-e416-4699-bed8-71a144239c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:12.164727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a3bd-e416-4699-bed8-71a144239c2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:12.165634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532a3bd-e416-4699-bed8-71a144239c2c None admin] RBAC: Adding query filter params (role.id=92af8b532e69461b9088cc236a91aa39, user.id=5a8b44cd796e4e1d8971b1b732f35f69, scope.system=all) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:12.166153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532a3bd-e416-4699-bed8-71a144239c2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:12.167931 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532a3bd-e416-4699-bed8-71a144239c2c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:12.187731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 121/255] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Oct 12 12:20:11 2022] GET /identity/v3/role_assignments?user.id=5a8b44cd796e4e1d8971b1b732f35f69&scope.system=all&role.id=92af8b532e69461b9088cc236a91aa39 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:13.688215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dc61c-f1a9-4db8-b488-6b2c7d4e3b27 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:13.688741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dc61c-f1a9-4db8-b488-6b2c7d4e3b27 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:13.688741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dc61c-f1a9-4db8-b488-6b2c7d4e3b27 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:13.690147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 135/256] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:13.695475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19650b-4549-4848-b6d2-d59a4d1aaf23 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:13.695885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19650b-4549-4848-b6d2-d59a4d1aaf23 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:13.696029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19650b-4549-4848-b6d2-d59a4d1aaf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:13.717205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1f19650b-4549-4848-b6d2-d59a4d1aaf23 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:13.718591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1f19650b-4549-4848-b6d2-d59a4d1aaf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:13.787837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 122/257] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:20:14.041508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80205d-d4c7-4f76-a801-27110ee0d0e6 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.041794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80205d-d4c7-4f76-a801-27110ee0d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.042107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80205d-d4c7-4f76-a801-27110ee0d0e6 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.043473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 136/258] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:14.056012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ae690-b82c-49bb-8de6-8c7a670998ce None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.056301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ae690-b82c-49bb-8de6-8c7a670998ce None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.056571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ae690-b82c-49bb-8de6-8c7a670998ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.091481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b60ae690-b82c-49bb-8de6-8c7a670998ce None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:14.092689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b60ae690-b82c-49bb-8de6-8c7a670998ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:14.108497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 123/259] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:20:14.120165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80205d-d4c7-4f76-a801-27110ee0d0e6 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.133474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80205d-d4c7-4f76-a801-27110ee0d0e6 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.137840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d229d-9e7e-44e1-b911-5bd9bd8738f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.138555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d229d-9e7e-44e1-b911-5bd9bd8738f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.138840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d229d-9e7e-44e1-b911-5bd9bd8738f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.139112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d229d-9e7e-44e1-b911-5bd9bd8738f7 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.154478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d229d-9e7e-44e1-b911-5bd9bd8738f7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-909d229d-9e7e-44e1-b911-5bd9bd8738f7 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:14.159426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:14.160290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:14.161055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 137/260] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:14.167722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ae690-b82c-49bb-8de6-8c7a670998ce None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.181852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ae690-b82c-49bb-8de6-8c7a670998ce None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.190258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.190878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.191156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.191422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.192091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:14.192394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.194255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:20:14.194255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 12 12:20:14.194255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: Oct 12 12:20:14.195165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 124/261] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:14.212683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d229d-9e7e-44e1-b911-5bd9bd8738f7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.224473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d229d-9e7e-44e1-b911-5bd9bd8738f7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.228944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.229680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.229990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.230268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] PATH_INFO: `/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.238259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] RBAC: Authorizing `identity:get_group(group_id=8650fd536df34e6ab8cd2bbe7d29aa87)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.240110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:20:14.240110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:20:14.240110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:20:14.241330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.243848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 138/262] 10.222.0.44 () {56 vars in 1201 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:14.249253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.260043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f26514-ee55-4f2d-b953-a45b9c1776b1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.264566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c36bb-0b26-46ca-803a-13ba6a21fda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.265249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c36bb-0b26-46ca-803a-13ba6a21fda2 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.265531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c36bb-0b26-46ca-803a-13ba6a21fda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.265804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c36bb-0b26-46ca-803a-13ba6a21fda2 admin admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.267059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c36bb-0b26-46ca-803a-13ba6a21fda2 admin admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.269286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c36bb-0b26-46ca-803a-13ba6a21fda2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.271009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 125/263] 10.222.0.44 () {56 vars in 1207 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:14.276746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.288906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a95cf1-8c39-45eb-bf39-a5f2c8ad1472 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.294478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.295149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.295426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.295749 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383/groups/8650fd536df34e6ab8cd2bbe7d29aa87/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.299504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bc485e87ac48e7ae6a07d85c170383, group_id=8650fd536df34e6ab8cd2bbe7d29aa87, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.300650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:20:14.300650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:20:14.300650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:20:14.302792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.432594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 139/264] 10.222.0.44 () {58 vars in 1463 bytes} [Wed Oct 12 12:20:14 2022] PUT /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383/groups/8650fd536df34e6ab8cd2bbe7d29aa87/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:14.545557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c36bb-0b26-46ca-803a-13ba6a21fda2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.555829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c36bb-0b26-46ca-803a-13ba6a21fda2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.622690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba32bd4-0074-4c36-b273-2ccd7aa6c608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.623354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba32bd4-0074-4c36-b273-2ccd7aa6c608 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.623637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba32bd4-0074-4c36-b273-2ccd7aa6c608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.623911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba32bd4-0074-4c36-b273-2ccd7aa6c608 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.639505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba32bd4-0074-4c36-b273-2ccd7aa6c608 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-7ba32bd4-0074-4c36-b273-2ccd7aa6c608 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:14.645462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:14.646537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:14.647315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 126/265] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 102 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:14.653384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.664993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d41f73-3ed3-4747-a99e-85bbfbef1158 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.669912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.670561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.670840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.671109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.671783 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:14.672087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.673158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:20:14.673158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:20:14.673158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:20:14.674075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.684429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 140/266] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:14.690226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba32bd4-0074-4c36-b273-2ccd7aa6c608 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.700877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba32bd4-0074-4c36-b273-2ccd7aa6c608 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.705510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.706229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.706506 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.707225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] PATH_INFO: `/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.708383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] RBAC: Authorizing `identity:get_group(group_id=8650fd536df34e6ab8cd2bbe7d29aa87)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.709417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:20:14.709417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 12 12:20:14.709417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: Oct 12 12:20:14.710737 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.712703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 127/267] 10.222.0.44 () {56 vars in 1201 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87 => generated 226 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:14.718584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.731677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f3c91-b629-4b42-abdc-22fc07d641cb admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.736587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9133c97-3257-4ee0-93ff-8aa4391f9fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.737296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9133c97-3257-4ee0-93ff-8aa4391f9fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.737581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9133c97-3257-4ee0-93ff-8aa4391f9fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.737850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9133c97-3257-4ee0-93ff-8aa4391f9fdf admin admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.739045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9133c97-3257-4ee0-93ff-8aa4391f9fdf admin admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.741300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9133c97-3257-4ee0-93ff-8aa4391f9fdf admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.743966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 141/268] 10.222.0.44 () {56 vars in 1207 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:14.749826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.761257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997259e-2c4e-465d-92fd-e80fdfe8ce92 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.765981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9834-73fd-479f-bf68-40a3af7d1fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.766661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9834-73fd-479f-bf68-40a3af7d1fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.766993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9834-73fd-479f-bf68-40a3af7d1fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.767274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9834-73fd-479f-bf68-40a3af7d1fd5 admin admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383/groups/8650fd536df34e6ab8cd2bbe7d29aa87/roles/5c4536b29c2a4ad2a8d05d6e94e6a186` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.769711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9834-73fd-479f-bf68-40a3af7d1fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bc485e87ac48e7ae6a07d85c170383, group_id=8650fd536df34e6ab8cd2bbe7d29aa87, role_id=5c4536b29c2a4ad2a8d05d6e94e6a186)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.770844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-e60f9834-73fd-479f-bf68-40a3af7d1fd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:20:14.770844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 12 12:20:14.770844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: Oct 12 12:20:14.772963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9834-73fd-479f-bf68-40a3af7d1fd5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.960450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 128/269] 10.222.0.44 () {58 vars in 1463 bytes} [Wed Oct 12 12:20:14 2022] PUT /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383/groups/8650fd536df34e6ab8cd2bbe7d29aa87/roles/5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:15.142201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89987-6ef9-4d79-bcf4-146a3598c1fe None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.142524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89987-6ef9-4d79-bcf4-146a3598c1fe None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.142756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b89987-6ef9-4d79-bcf4-146a3598c1fe None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.144384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 142/270] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:15.150232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225d25e-b857-4b86-bd8d-eab2d295c1f6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.150528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225d25e-b857-4b86-bd8d-eab2d295c1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.150807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225d25e-b857-4b86-bd8d-eab2d295c1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.174882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5225d25e-b857-4b86-bd8d-eab2d295c1f6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:15.176149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5225d25e-b857-4b86-bd8d-eab2d295c1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:15.189233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:15.244208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 129/271] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:20:15.563861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa6be-a78e-44dd-973d-c5836927d9e4 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.564231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa6be-a78e-44dd-973d-c5836927d9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.564408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa6be-a78e-44dd-973d-c5836927d9e4 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.566005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 143/272] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:15.572099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7a4b9-2cd3-4312-9390-e41393b76704 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.572388 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7a4b9-2cd3-4312-9390-e41393b76704 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.572668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7a4b9-2cd3-4312-9390-e41393b76704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.592245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0fd7a4b9-2cd3-4312-9390-e41393b76704 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:15.593429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0fd7a4b9-2cd3-4312-9390-e41393b76704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:15.674228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 130/273] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:15.730102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fa6be-a78e-44dd-973d-c5836927d9e4 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:15.748019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fa6be-a78e-44dd-973d-c5836927d9e4 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:15.752801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0591dff-4212-4c73-bb42-c700f956e5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:15.753480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0591dff-4212-4c73-bb42-c700f956e5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.753766 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0591dff-4212-4c73-bb42-c700f956e5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.754122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0591dff-4212-4c73-bb42-c700f956e5b8 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.767931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0591dff-4212-4c73-bb42-c700f956e5b8 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-a0591dff-4212-4c73-bb42-c700f956e5b8 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:15.773490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:15.774425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:15.775212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 144/274] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Oct 12 12:20:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:15.782077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7a4b9-2cd3-4312-9390-e41393b76704 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:15.793533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7a4b9-2cd3-4312-9390-e41393b76704 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:15.798576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:15.799225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.799503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.799774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.800495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:15.800798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:15.802521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:15.818119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 131/275] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:20:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:15.824354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0591dff-4212-4c73-bb42-c700f956e5b8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:15.831904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:15.836010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0591dff-4212-4c73-bb42-c700f956e5b8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:15.840693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773cd06-7709-4c30-9031-4c0e4dcecbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:15.841384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773cd06-7709-4c30-9031-4c0e4dcecbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.841666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773cd06-7709-4c30-9031-4c0e4dcecbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.842027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773cd06-7709-4c30-9031-4c0e4dcecbf2 admin admin] PATH_INFO: `/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.844158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773cd06-7709-4c30-9031-4c0e4dcecbf2 admin admin] RBAC: Authorizing `identity:get_group(group_id=8650fd536df34e6ab8cd2bbe7d29aa87)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:15.852380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773cd06-7709-4c30-9031-4c0e4dcecbf2 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:15.852380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 145/276] 10.222.0.44 () {56 vars in 1201 bytes} [Wed Oct 12 12:20:15 2022] GET /identity/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87 => generated 226 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:15.854471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:15.865674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66717f54-11e3-4dac-b1c2-d6831c51daad admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:15.873971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483e72-edf8-4f13-b981-0312d7ab1ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:15.874623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483e72-edf8-4f13-b981-0312d7ab1ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.874900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483e72-edf8-4f13-b981-0312d7ab1ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.875170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483e72-edf8-4f13-b981-0312d7ab1ab6 admin admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.876408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94483e72-edf8-4f13-b981-0312d7ab1ab6 admin admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:15.878662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94483e72-edf8-4f13-b981-0312d7ab1ab6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:15.882162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 132/277] 10.222.0.44 () {56 vars in 1207 bytes} [Wed Oct 12 12:20:15 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:15.886582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773cd06-7709-4c30-9031-4c0e4dcecbf2 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:15.896926 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773cd06-7709-4c30-9031-4c0e4dcecbf2 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:15.901849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa4c17a-cf1a-43b2-9877-c5584f734435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:15.902551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4c17a-cf1a-43b2-9877-c5584f734435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:15.902913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4c17a-cf1a-43b2-9877-c5584f734435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:15.903192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa4c17a-cf1a-43b2-9877-c5584f734435 admin admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae/groups/8650fd536df34e6ab8cd2bbe7d29aa87/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:15.906551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa4c17a-cf1a-43b2-9877-c5584f734435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b46ee23713a44848ffd7947384566ae, group_id=8650fd536df34e6ab8cd2bbe7d29aa87, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:15.909349 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa4c17a-cf1a-43b2-9877-c5584f734435 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:15.989405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 146/278] 10.222.0.44 () {58 vars in 1463 bytes} [Wed Oct 12 12:20:15 2022] PUT /identity/v3/projects/8b46ee23713a44848ffd7947384566ae/groups/8650fd536df34e6ab8cd2bbe7d29aa87/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:16.345546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f539-c335-4ca0-a299-e596c819f43c None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.345888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f539-c335-4ca0-a299-e596c819f43c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.346028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f539-c335-4ca0-a299-e596c819f43c None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.347459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 133/279] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:16.354157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac5461-163f-47ba-b8ca-9d25267d7932 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.354436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac5461-163f-47ba-b8ca-9d25267d7932 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.354714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac5461-163f-47ba-b8ca-9d25267d7932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.383153 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b2ac5461-163f-47ba-b8ca-9d25267d7932 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:16.384408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b2ac5461-163f-47ba-b8ca-9d25267d7932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:16.416814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f539-c335-4ca0-a299-e596c819f43c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:16.462702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 147/280] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:20:16.518152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:16.547421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f539-c335-4ca0-a299-e596c819f43c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:16.588282 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7bbee-e35f-4516-b688-2d69b39272fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:16.588982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bbee-e35f-4516-b688-2d69b39272fb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.589267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bbee-e35f-4516-b688-2d69b39272fb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.589536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7bbee-e35f-4516-b688-2d69b39272fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.600621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7bbee-e35f-4516-b688-2d69b39272fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-c5d7bbee-e35f-4516-b688-2d69b39272fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:16.608122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:16.609077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:16.610309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 134/281] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:16.615725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac5461-163f-47ba-b8ca-9d25267d7932 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:16.739337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac5461-163f-47ba-b8ca-9d25267d7932 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:16.775758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10db360-a100-4367-b152-48283a61a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:16.776374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10db360-a100-4367-b152-48283a61a0ee None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.776651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10db360-a100-4367-b152-48283a61a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.776922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10db360-a100-4367-b152-48283a61a0ee None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.777581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10db360-a100-4367-b152-48283a61a0ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:16.777883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10db360-a100-4367-b152-48283a61a0ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:16.779277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10db360-a100-4367-b152-48283a61a0ee None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:16.780667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:16.790619 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 148/282] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:16.796616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7bbee-e35f-4516-b688-2d69b39272fb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:16.858060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81c743-8ac2-4871-a44e-d405cf1b5921 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.858346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81c743-8ac2-4871-a44e-d405cf1b5921 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.858528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81c743-8ac2-4871-a44e-d405cf1b5921 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.860122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 149/283] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:16.864582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf29b71-e661-47d3-9b6d-8e79111c0fb0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.864860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf29b71-e661-47d3-9b6d-8e79111c0fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.865135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf29b71-e661-47d3-9b6d-8e79111c0fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.894560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ecf29b71-e661-47d3-9b6d-8e79111c0fb0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:16.896404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ecf29b71-e661-47d3-9b6d-8e79111c0fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:16.909381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 150/284] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:20:16.911042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf29b71-e661-47d3-9b6d-8e79111c0fb0 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:16.931497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf29b71-e661-47d3-9b6d-8e79111c0fb0 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:16.940899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc49f3b-a3c3-4a1b-8af9-60d40bf16c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:16.942819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc49f3b-a3c3-4a1b-8af9-60d40bf16c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.943102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc49f3b-a3c3-4a1b-8af9-60d40bf16c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.943372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc49f3b-a3c3-4a1b-8af9-60d40bf16c88 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.956655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7bbee-e35f-4516-b688-2d69b39272fb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:16.970152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc49f3b-a3c3-4a1b-8af9-60d40bf16c88 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-5cc49f3b-a3c3-4a1b-8af9-60d40bf16c88 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:16.979494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:16.980387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:16.981188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:16.983737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 151/285] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Oct 12 12:20:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 74 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:16.990230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc49f3b-a3c3-4a1b-8af9-60d40bf16c88 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.002198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc49f3b-a3c3-4a1b-8af9-60d40bf16c88 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.007197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.008665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.009368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.009787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.010646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:17.011093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.012894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.026255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 152/286] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Oct 12 12:20:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.035908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.044517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299ad63-846e-48e0-90ab-5e638cd9e28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.045170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299ad63-846e-48e0-90ab-5e638cd9e28c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.045452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299ad63-846e-48e0-90ab-5e638cd9e28c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.045779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299ad63-846e-48e0-90ab-5e638cd9e28c None admin] PATH_INFO: `/v3/users/6f354288fac64a8c81a360c3b220805f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.046901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299ad63-846e-48e0-90ab-5e638cd9e28c None admin] RBAC: Authorizing `identity:get_user(user_id=6f354288fac64a8c81a360c3b220805f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.048477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299ad63-846e-48e0-90ab-5e638cd9e28c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.048878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095cfc-51ea-4aab-ac91-9985194ec8b6 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.054391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 135/287] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:16 2022] GET /identity/v3/users/6f354288fac64a8c81a360c3b220805f => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.061736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299ad63-846e-48e0-90ab-5e638cd9e28c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.066380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e2b13-a2ee-4c6a-85bf-375f4eb2376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.073980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e2b13-a2ee-4c6a-85bf-375f4eb2376e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.074128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:17.074204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e2b13-a2ee-4c6a-85bf-375f4eb2376e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.074204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e2b13-a2ee-4c6a-85bf-375f4eb2376e admin admin] PATH_INFO: `/v3/groups/9cff2218b9f44d899b7c5d3142fcafd2` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.086695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e2b13-a2ee-4c6a-85bf-375f4eb2376e admin admin] RBAC: Authorizing `identity:get_group(group_id=9cff2218b9f44d899b7c5d3142fcafd2)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.088737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e2b13-a2ee-4c6a-85bf-375f4eb2376e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.092690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 153/288] 10.222.0.44 () {56 vars in 1201 bytes} [Wed Oct 12 12:20:17 2022] GET /identity/v3/groups/9cff2218b9f44d899b7c5d3142fcafd2 => generated 229 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.096168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e2b13-a2ee-4c6a-85bf-375f4eb2376e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.116368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e2b13-a2ee-4c6a-85bf-375f4eb2376e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.124899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb091ae6-3292-4895-a0af-90bcd02164ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.125812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb091ae6-3292-4895-a0af-90bcd02164ac admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.126332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb091ae6-3292-4895-a0af-90bcd02164ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.126794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb091ae6-3292-4895-a0af-90bcd02164ac admin admin] PATH_INFO: `/v3/projects/5655a305f69b4cb0901b114b25e5c478` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.130487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb091ae6-3292-4895-a0af-90bcd02164ac admin admin] RBAC: Authorizing `identity:get_project(project_id=5655a305f69b4cb0901b114b25e5c478)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.132498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb091ae6-3292-4895-a0af-90bcd02164ac admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.135235 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 154/289] 10.222.0.44 () {56 vars in 1207 bytes} [Wed Oct 12 12:20:17 2022] GET /identity/v3/projects/5655a305f69b4cb0901b114b25e5c478 => generated 344 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.140760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb091ae6-3292-4895-a0af-90bcd02164ac admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.153471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb091ae6-3292-4895-a0af-90bcd02164ac admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.162555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d33096-8af1-4fe1-a005-7c261c703cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.163457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33096-8af1-4fe1-a005-7c261c703cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.163930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33096-8af1-4fe1-a005-7c261c703cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.164432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33096-8af1-4fe1-a005-7c261c703cd9 admin admin] PATH_INFO: `/v3/projects/5655a305f69b4cb0901b114b25e5c478/groups/9cff2218b9f44d899b7c5d3142fcafd2/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.169872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d33096-8af1-4fe1-a005-7c261c703cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5655a305f69b4cb0901b114b25e5c478, group_id=9cff2218b9f44d899b7c5d3142fcafd2, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.172872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d33096-8af1-4fe1-a005-7c261c703cd9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.207756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299ad63-846e-48e0-90ab-5e638cd9e28c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.248499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.249114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.249395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.249669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.250479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] RBAC: Adding query filter params (role.id=672667b4e6d14a17ac7cb28a93d1c48e, user.id=6f354288fac64a8c81a360c3b220805f, scope.system=all) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:17.250796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.252383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.267666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 136/290] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Oct 12 12:20:17 2022] GET /identity/v3/role_assignments?user.id=6f354288fac64a8c81a360c3b220805f&scope.system=all&role.id=672667b4e6d14a17ac7cb28a93d1c48e => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.269690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 155/291] 10.222.0.44 () {58 vars in 1463 bytes} [Wed Oct 12 12:20:17 2022] PUT /identity/v3/projects/5655a305f69b4cb0901b114b25e5c478/groups/9cff2218b9f44d899b7c5d3142fcafd2/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:20:17.324048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.401542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3662c64-d527-4408-b633-0b4611f5cb47 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.406147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0358ef-e4b3-45df-9d1c-617fa421e78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.406863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0358ef-e4b3-45df-9d1c-617fa421e78c admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.407149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0358ef-e4b3-45df-9d1c-617fa421e78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.407422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0358ef-e4b3-45df-9d1c-617fa421e78c admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.419661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:17.426644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0358ef-e4b3-45df-9d1c-617fa421e78c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-7d0358ef-e4b3-45df-9d1c-617fa421e78c admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:17.432992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:17.438560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:17.439350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 137/292] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Oct 12 12:20:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 111 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:17.441410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d33096-8af1-4fe1-a005-7c261c703cd9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.452822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d33096-8af1-4fe1-a005-7c261c703cd9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.457885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.458591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.458925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.459198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.459878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:17.460176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.461966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.472105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 156/293] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Oct 12 12:20:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.478826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0358ef-e4b3-45df-9d1c-617fa421e78c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.489428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0358ef-e4b3-45df-9d1c-617fa421e78c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.497228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec0e84-50bd-4049-a9c7-b1bf8cff1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.497960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec0e84-50bd-4049-a9c7-b1bf8cff1dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.498312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec0e84-50bd-4049-a9c7-b1bf8cff1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.498519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec0e84-50bd-4049-a9c7-b1bf8cff1dd9 admin admin] PATH_INFO: `/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.499800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec0e84-50bd-4049-a9c7-b1bf8cff1dd9 admin admin] RBAC: Authorizing `identity:get_group(group_id=8650fd536df34e6ab8cd2bbe7d29aa87)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.502033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec0e84-50bd-4049-a9c7-b1bf8cff1dd9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.503849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 138/294] 10.222.0.44 () {56 vars in 1201 bytes} [Wed Oct 12 12:20:17 2022] GET /identity/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87 => generated 226 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.509792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.520982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7b8db-7e0d-4f8a-a378-62501564949a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.525898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd044877-6955-417c-967a-09b1c1326185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.526579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd044877-6955-417c-967a-09b1c1326185 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.526910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd044877-6955-417c-967a-09b1c1326185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.527185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd044877-6955-417c-967a-09b1c1326185 admin admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.528386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd044877-6955-417c-967a-09b1c1326185 admin admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.530568 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd044877-6955-417c-967a-09b1c1326185 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.532728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 157/295] 10.222.0.44 () {56 vars in 1207 bytes} [Wed Oct 12 12:20:17 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.538809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec0e84-50bd-4049-a9c7-b1bf8cff1dd9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.550272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec0e84-50bd-4049-a9c7-b1bf8cff1dd9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.555747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb338-d366-48f9-9b8f-091d0ae8fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.556495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb338-d366-48f9-9b8f-091d0ae8fdb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.556777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb338-d366-48f9-9b8f-091d0ae8fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.557097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb338-d366-48f9-9b8f-091d0ae8fdb1 admin admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae/groups/8650fd536df34e6ab8cd2bbe7d29aa87/roles/5c4536b29c2a4ad2a8d05d6e94e6a186` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.559470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb338-d366-48f9-9b8f-091d0ae8fdb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b46ee23713a44848ffd7947384566ae, group_id=8650fd536df34e6ab8cd2bbe7d29aa87, role_id=5c4536b29c2a4ad2a8d05d6e94e6a186)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.562371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb338-d366-48f9-9b8f-091d0ae8fdb1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.675299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 139/296] 10.222.0.44 () {58 vars in 1463 bytes} [Wed Oct 12 12:20:17 2022] PUT /identity/v3/projects/8b46ee23713a44848ffd7947384566ae/groups/8650fd536df34e6ab8cd2bbe7d29aa87/roles/5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:18.011422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be573eb-50e4-4d44-8624-6875c37118b0 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:18.011911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be573eb-50e4-4d44-8624-6875c37118b0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:18.011911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be573eb-50e4-4d44-8624-6875c37118b0 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:18.013560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 158/297] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:18.018622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3457af-b187-4292-a0f5-dba3486b449a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:18.019479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3457af-b187-4292-a0f5-dba3486b449a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:18.019758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3457af-b187-4292-a0f5-dba3486b449a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:18.035028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfacf5-76d9-4e97-b5f4-d4c85e27a108 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:18.035651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfacf5-76d9-4e97-b5f4-d4c85e27a108 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:18.036126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfacf5-76d9-4e97-b5f4-d4c85e27a108 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:18.037690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 159/298] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:18.042969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2e3457af-b187-4292-a0f5-dba3486b449a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:18.044465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2e3457af-b187-4292-a0f5-dba3486b449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:18.045034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17c67-f343-4707-8b4b-bf47507ecfa9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:18.045516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17c67-f343-4707-8b4b-bf47507ecfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:18.046036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17c67-f343-4707-8b4b-bf47507ecfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:18.076249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-cfd17c67-f343-4707-8b4b-bf47507ecfa9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:18.077723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-cfd17c67-f343-4707-8b4b-bf47507ecfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:18.125248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 140/299] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:18.187147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 160/300] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:18.482945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3457af-b187-4292-a0f5-dba3486b449a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:18.509625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd17c67-f343-4707-8b4b-bf47507ecfa9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:18.592139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3457af-b187-4292-a0f5-dba3486b449a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:18.606826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:18.623091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:18.662023 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6699227-39d5-4eaf-8774-894ea86226a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:18.663153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6699227-39d5-4eaf-8774-894ea86226a9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:18.663433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6699227-39d5-4eaf-8774-894ea86226a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:18.663705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6699227-39d5-4eaf-8774-894ea86226a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:18.671491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd17c67-f343-4707-8b4b-bf47507ecfa9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:18.688793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6699227-39d5-4eaf-8774-894ea86226a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-c6699227-39d5-4eaf-8774-894ea86226a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:18.696000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:18.696908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:18.698288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 141/301] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:20:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:18.703032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6699227-39d5-4eaf-8774-894ea86226a9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:18.740866 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffa8c2-9225-4e05-b955-481ff50d3ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:18.741472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa8c2-9225-4e05-b955-481ff50d3ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:18.741755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa8c2-9225-4e05-b955-481ff50d3ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:18.742044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa8c2-9225-4e05-b955-481ff50d3ae3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:18.766579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffa8c2-9225-4e05-b955-481ff50d3ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-27ffa8c2-9225-4e05-b955-481ff50d3ae3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:18.783149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:18.784115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:18.785913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 161/302] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:18.793611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffa8c2-9225-4e05-b955-481ff50d3ae3 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:18.900234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6699227-39d5-4eaf-8774-894ea86226a9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:18.968771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffa8c2-9225-4e05-b955-481ff50d3ae3 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:19.006409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:19.011976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:19.012804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.022038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.022038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.022038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:19.022038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:19.022038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:19.025727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:19.026371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.026653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.026928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.027594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] RBAC: Adding query filter params (name=member) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:19.027899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:19.029963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:19.036443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 142/303] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:19.042190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:19.043301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 162/304] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:19.049879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:19.150311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:19.217756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c302615-e597-4482-8cd3-e9e40a0e59be None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:19.272051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f1a55-b0cf-4309-863e-5be989b8323d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:19.291763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16619772-e6c6-4655-b3ca-8dc9a5d1fde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:19.292729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16619772-e6c6-4655-b3ca-8dc9a5d1fde3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.293242 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16619772-e6c6-4655-b3ca-8dc9a5d1fde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.293713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16619772-e6c6-4655-b3ca-8dc9a5d1fde3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.339887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f17090-f977-45e4-9932-fce4b38940a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:19.340873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17090-f977-45e4-9932-fce4b38940a3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.341354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17090-f977-45e4-9932-fce4b38940a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.341848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f17090-f977-45e4-9932-fce4b38940a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.382035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:19.408047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16619772-e6c6-4655-b3ca-8dc9a5d1fde3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:19.409543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16619772-e6c6-4655-b3ca-8dc9a5d1fde3 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:19.437479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f17090-f977-45e4-9932-fce4b38940a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:19.438977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f17090-f977-45e4-9932-fce4b38940a3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:19.479904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:19.541202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 163/305] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 12:20:19.543515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbebb4-064e-4f7b-99fa-e46b4c6d4deb None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.543805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbebb4-064e-4f7b-99fa-e46b4c6d4deb None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.544078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbebb4-064e-4f7b-99fa-e46b4c6d4deb None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.546084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 164/306] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:19.547050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbebb4-064e-4f7b-99fa-e46b4c6d4deb None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:19.561015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 143/307] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 12:20:19.562781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e0f3-c37d-40af-a259-0ca3f127a3e2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.563139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e0f3-c37d-40af-a259-0ca3f127a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.563419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9e0f3-c37d-40af-a259-0ca3f127a3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.585407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f1b9e0f3-c37d-40af-a259-0ca3f127a3e2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:19.587853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f1b9e0f3-c37d-40af-a259-0ca3f127a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:19.658916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:19.672268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 144/308] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:19.673123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9e0f3-c37d-40af-a259-0ca3f127a3e2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:19.695202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbebb4-064e-4f7b-99fa-e46b4c6d4deb None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:19.755388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc87f-28c6-48a8-8e86-bb8afddd461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:19.832506 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:19.832664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc87f-28c6-48a8-8e86-bb8afddd461a None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.832664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc87f-28c6-48a8-8e86-bb8afddd461a None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.832664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc87f-28c6-48a8-8e86-bb8afddd461a None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.832664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc87f-28c6-48a8-8e86-bb8afddd461a None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:19.832664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc87f-28c6-48a8-8e86-bb8afddd461a None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:19.832664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 165/309] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:19 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:19.833064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc87f-28c6-48a8-8e86-bb8afddd461a None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:19.833215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9e0f3-c37d-40af-a259-0ca3f127a3e2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:19.833215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689869c-1bf0-4aaf-ab87-0186d6aa8da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:19.833215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689869c-1bf0-4aaf-ab87-0186d6aa8da4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.833215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689869c-1bf0-4aaf-ab87-0186d6aa8da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.833479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689869c-1bf0-4aaf-ab87-0186d6aa8da4 None admin] PATH_INFO: `/v3/projects/43bc485e87ac48e7ae6a07d85c170383` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.833479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689869c-1bf0-4aaf-ab87-0186d6aa8da4 None admin] RBAC: Authorizing `identity:get_project(project_id=43bc485e87ac48e7ae6a07d85c170383)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:19.833918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689869c-1bf0-4aaf-ab87-0186d6aa8da4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:19.835428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 145/310] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:19 2022] GET /identity/v3/projects/43bc485e87ac48e7ae6a07d85c170383 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:19.843157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689869c-1bf0-4aaf-ab87-0186d6aa8da4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:19.968862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc87f-28c6-48a8-8e86-bb8afddd461a None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.044875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689869c-1bf0-4aaf-ab87-0186d6aa8da4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.045541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f511f-7093-4610-b146-59ab4119dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.045541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f511f-7093-4610-b146-59ab4119dd78 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.045541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f511f-7093-4610-b146-59ab4119dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.045541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f511f-7093-4610-b146-59ab4119dd78 None admin] PATH_INFO: `/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.053979 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f511f-7093-4610-b146-59ab4119dd78 None admin] RBAC: Authorizing `identity:get_group(group_id=8650fd536df34e6ab8cd2bbe7d29aa87)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.053979 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f511f-7093-4610-b146-59ab4119dd78 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.055175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 166/311] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Oct 12 12:20:19 2022] GET /identity/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:20.061571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f511f-7093-4610-b146-59ab4119dd78 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.090031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20551e-d77a-4d3a-bdd3-0f1769904f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.090031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20551e-d77a-4d3a-bdd3-0f1769904f74 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.090031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20551e-d77a-4d3a-bdd3-0f1769904f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.090031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20551e-d77a-4d3a-bdd3-0f1769904f74 None admin] PATH_INFO: `/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.090395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20551e-d77a-4d3a-bdd3-0f1769904f74 None admin] RBAC: Authorizing `identity:get_group(group_id=8650fd536df34e6ab8cd2bbe7d29aa87)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.090395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20551e-d77a-4d3a-bdd3-0f1769904f74 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.091659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 146/312] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Oct 12 12:20:19 2022] GET /identity/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:20.097414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20551e-d77a-4d3a-bdd3-0f1769904f74 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.102202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.114916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.226756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20551e-d77a-4d3a-bdd3-0f1769904f74 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.232569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f511f-7093-4610-b146-59ab4119dd78 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.264438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.265046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.265348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.265616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.267740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] RBAC: Adding query filter params (group.id=8650fd536df34e6ab8cd2bbe7d29aa87, role.id=5c4536b29c2a4ad2a8d05d6e94e6a186, scope.project.id=43bc485e87ac48e7ae6a07d85c170383) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:20.267740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.268470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.275261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 147/313] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Oct 12 12:20:20 2022] GET /identity/v3/role_assignments?group.id=8650fd536df34e6ab8cd2bbe7d29aa87&scope.project.id=43bc485e87ac48e7ae6a07d85c170383&role.id=5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:20.276083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.277142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93cd95-6140-40b0-a330-b2da1bdd7e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.278049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93cd95-6140-40b0-a330-b2da1bdd7e62 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.278603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93cd95-6140-40b0-a330-b2da1bdd7e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.279076 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93cd95-6140-40b0-a330-b2da1bdd7e62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.280025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93cd95-6140-40b0-a330-b2da1bdd7e62 None admin] RBAC: Adding query filter params (group.id=8650fd536df34e6ab8cd2bbe7d29aa87, role.id=92af8b532e69461b9088cc236a91aa39, scope.project.id=43bc485e87ac48e7ae6a07d85c170383) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:20.280543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93cd95-6140-40b0-a330-b2da1bdd7e62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.282282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93cd95-6140-40b0-a330-b2da1bdd7e62 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.289797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 167/314] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Oct 12 12:20:20 2022] GET /identity/v3/role_assignments?group.id=8650fd536df34e6ab8cd2bbe7d29aa87&scope.project.id=43bc485e87ac48e7ae6a07d85c170383&role.id=92af8b532e69461b9088cc236a91aa39 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:20.349040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfac7c-2045-4174-8e69-9ee65b060d5e None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.349671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfac7c-2045-4174-8e69-9ee65b060d5e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.350169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfac7c-2045-4174-8e69-9ee65b060d5e None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.352346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 168/315] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:20.371503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525bf39-eeb0-49ec-a002-fa826921fcd4 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.372127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525bf39-eeb0-49ec-a002-fa826921fcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.372617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525bf39-eeb0-49ec-a002-fa826921fcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.382370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.410175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7757f8-4e2a-4d1a-9bdc-f38972bee376 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.410730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-6525bf39-eeb0-49ec-a002-fa826921fcd4 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:20.412298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6525bf39-eeb0-49ec-a002-fa826921fcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:20.452751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac50000-0c53-4f1d-bcb1-89945e257205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.453392 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac50000-0c53-4f1d-bcb1-89945e257205 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.453675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac50000-0c53-4f1d-bcb1-89945e257205 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.453959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac50000-0c53-4f1d-bcb1-89945e257205 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.473373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac50000-0c53-4f1d-bcb1-89945e257205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-bac50000-0c53-4f1d-bcb1-89945e257205 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:20.488817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:20.489819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:20:20.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:20.493392 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 148/316] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:20.514198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac50000-0c53-4f1d-bcb1-89945e257205 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.563029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 169/317] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:20.564410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2970c69-e71a-40f9-a35a-047065d69713 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.564701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2970c69-e71a-40f9-a35a-047065d69713 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.564980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2970c69-e71a-40f9-a35a-047065d69713 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.567911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 170/318] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:20.573629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afff85-393b-4b1c-9f81-de1b6f94eb92 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.573999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afff85-393b-4b1c-9f81-de1b6f94eb92 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.574277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afff85-393b-4b1c-9f81-de1b6f94eb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.594995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-39afff85-393b-4b1c-9f81-de1b6f94eb92 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:20.596203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-39afff85-393b-4b1c-9f81-de1b6f94eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:20.605499 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.658198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac50000-0c53-4f1d-bcb1-89945e257205 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.693715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.699250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 171/319] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:20.703358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.704044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.704505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.704617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.706629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:20.706629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.707123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.716628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 149/320] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:20:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:20.722328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afff85-393b-4b1c-9f81-de1b6f94eb92 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.827400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afff85-393b-4b1c-9f81-de1b6f94eb92 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.864129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020cc92-9e24-43b8-ac75-a32ad86ee20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.864720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020cc92-9e24-43b8-ac75-a32ad86ee20b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.865005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020cc92-9e24-43b8-ac75-a32ad86ee20b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.865279 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020cc92-9e24-43b8-ac75-a32ad86ee20b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.877573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.899833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.946719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020cc92-9e24-43b8-ac75-a32ad86ee20b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.948594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020cc92-9e24-43b8-ac75-a32ad86ee20b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:21.021316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe31ce-dfa3-4d6d-91f5-1ff008b5baf6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.060603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 172/321] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:21.061099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180844c-5f1c-41fb-87a1-8c0236e292c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.061443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180844c-5f1c-41fb-87a1-8c0236e292c5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.061514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020cc92-9e24-43b8-ac75-a32ad86ee20b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.061598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180844c-5f1c-41fb-87a1-8c0236e292c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.062031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180844c-5f1c-41fb-87a1-8c0236e292c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.077397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180844c-5f1c-41fb-87a1-8c0236e292c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1180844c-5f1c-41fb-87a1-8c0236e292c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:21.083339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:21.084404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:21.085198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 150/322] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:21.086999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180844c-5f1c-41fb-87a1-8c0236e292c5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.118327 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:21.146100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:21.182963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020cc92-9e24-43b8-ac75-a32ad86ee20b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.206133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180844c-5f1c-41fb-87a1-8c0236e292c5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.223340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e960fb-ed66-483e-9131-d9f5caf094ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.223974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e960fb-ed66-483e-9131-d9f5caf094ff None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.224249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e960fb-ed66-483e-9131-d9f5caf094ff None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.224518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e960fb-ed66-483e-9131-d9f5caf094ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.239630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e960fb-ed66-483e-9131-d9f5caf094ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-67e960fb-ed66-483e-9131-d9f5caf094ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:21.245127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:21.246053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:21.246851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 173/323] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:21.247514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e960fb-ed66-483e-9131-d9f5caf094ff None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.254791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c505c08-8f39-4618-bf1f-00e71ed28d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.255432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c505c08-8f39-4618-bf1f-00e71ed28d3d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.255713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c505c08-8f39-4618-bf1f-00e71ed28d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.256011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c505c08-8f39-4618-bf1f-00e71ed28d3d None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.257204 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c505c08-8f39-4618-bf1f-00e71ed28d3d None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:21.259225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c505c08-8f39-4618-bf1f-00e71ed28d3d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:21.260992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 151/324] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:21.262145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c505c08-8f39-4618-bf1f-00e71ed28d3d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.366790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e960fb-ed66-483e-9131-d9f5caf094ff None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.368662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:21.392202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c505c08-8f39-4618-bf1f-00e71ed28d3d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.407740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.409231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.409231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.409231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.409581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:21.409813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:21.411225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:21.413485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:21.422088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 174/325] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:21.431318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.438567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.439088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.439367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.439637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.440298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:21.440594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:21.442000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:21.453113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 152/326] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:21.454416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.542478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b176f03-9d4c-47f0-b480-7f5df9d3f182 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.578691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0814741-ba3b-45ee-839f-e4677c169296 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.580537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326166-eb6f-4f5d-9976-4aa4bb61829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.581656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326166-eb6f-4f5d-9976-4aa4bb61829c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.581656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326166-eb6f-4f5d-9976-4aa4bb61829c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.581847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326166-eb6f-4f5d-9976-4aa4bb61829c None admin] PATH_INFO: `/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.583382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b326166-eb6f-4f5d-9976-4aa4bb61829c None admin] RBAC: Authorizing `identity:get_group(group_id=8650fd536df34e6ab8cd2bbe7d29aa87)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:21.584410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b326166-eb6f-4f5d-9976-4aa4bb61829c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:21.586071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 175/327] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:21.586798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326166-eb6f-4f5d-9976-4aa4bb61829c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.619924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:21.624101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08970d21-41a6-40a6-9b7b-3e6664e83ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.625211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08970d21-41a6-40a6-9b7b-3e6664e83ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.625493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08970d21-41a6-40a6-9b7b-3e6664e83ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.625764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08970d21-41a6-40a6-9b7b-3e6664e83ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.678245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:21.696956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b326166-eb6f-4f5d-9976-4aa4bb61829c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.701085 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08970d21-41a6-40a6-9b7b-3e6664e83ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:21.702548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08970d21-41a6-40a6-9b7b-3e6664e83ce7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:21.732133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34380073-e2f8-4ba6-9bb8-6495498104c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.732731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34380073-e2f8-4ba6-9bb8-6495498104c9 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.733004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34380073-e2f8-4ba6-9bb8-6495498104c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.733270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34380073-e2f8-4ba6-9bb8-6495498104c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.797744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 153/328] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:21.800444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34380073-e2f8-4ba6-9bb8-6495498104c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:21.801896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34380073-e2f8-4ba6-9bb8-6495498104c9 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:21.802871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08970d21-41a6-40a6-9b7b-3e6664e83ce7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.832732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:21.879772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:21.899991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08970d21-41a6-40a6-9b7b-3e6664e83ce7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:21.901040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 176/329] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:21.901787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34380073-e2f8-4ba6-9bb8-6495498104c9 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:21.934360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:21.934959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:21.935234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:21.935504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:21.936221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] RBAC: Adding query filter params (group.id=8650fd536df34e6ab8cd2bbe7d29aa87, role.id=92af8b532e69461b9088cc236a91aa39, scope.project.id=8b46ee23713a44848ffd7947384566ae) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:21.936528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:21.937965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:21.943507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 154/330] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/role_assignments?group.id=8650fd536df34e6ab8cd2bbe7d29aa87&scope.project.id=8b46ee23713a44848ffd7947384566ae&role.id=92af8b532e69461b9088cc236a91aa39 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:21.944269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:22.014048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34380073-e2f8-4ba6-9bb8-6495498104c9 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:22.047389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:22.053225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6029613-6bb1-440d-96e3-892cfe19de72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:22.053874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6029613-6bb1-440d-96e3-892cfe19de72 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:22.054191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1d0b-77ec-4e10-8192-ec44f21495c8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:22.054311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6029613-6bb1-440d-96e3-892cfe19de72 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:22.054474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6029613-6bb1-440d-96e3-892cfe19de72 None admin] PATH_INFO: `/v3/projects/5655a305f69b4cb0901b114b25e5c478` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:22.056331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6029613-6bb1-440d-96e3-892cfe19de72 None admin] RBAC: Authorizing `identity:get_project(project_id=5655a305f69b4cb0901b114b25e5c478)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:22.058106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6029613-6bb1-440d-96e3-892cfe19de72 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:22.059912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 177/331] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/projects/5655a305f69b4cb0901b114b25e5c478 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:22.065547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6029613-6bb1-440d-96e3-892cfe19de72 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:22.103351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373ee16-a01b-4eef-ac24-7adcc41dd708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:22.103973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ee16-a01b-4eef-ac24-7adcc41dd708 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:22.104256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ee16-a01b-4eef-ac24-7adcc41dd708 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:22.104532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373ee16-a01b-4eef-ac24-7adcc41dd708 None admin] PATH_INFO: `/v3/projects/8b46ee23713a44848ffd7947384566ae` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:22.105700 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373ee16-a01b-4eef-ac24-7adcc41dd708 None admin] RBAC: Authorizing `identity:get_project(project_id=8b46ee23713a44848ffd7947384566ae)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:22.107202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373ee16-a01b-4eef-ac24-7adcc41dd708 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:22.109109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 155/332] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:21 2022] GET /identity/v3/projects/8b46ee23713a44848ffd7947384566ae => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:22.114424 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373ee16-a01b-4eef-ac24-7adcc41dd708 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:22.128474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:22.181345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6029613-6bb1-440d-96e3-892cfe19de72 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:22.219421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76a9ad-9211-4f55-9453-09d1f5017d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:22.219421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76a9ad-9211-4f55-9453-09d1f5017d02 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:22.219821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76a9ad-9211-4f55-9453-09d1f5017d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:22.220398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76a9ad-9211-4f55-9453-09d1f5017d02 None admin] PATH_INFO: `/v3/groups/9cff2218b9f44d899b7c5d3142fcafd2` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:22.223951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76a9ad-9211-4f55-9453-09d1f5017d02 None admin] RBAC: Authorizing `identity:get_group(group_id=9cff2218b9f44d899b7c5d3142fcafd2)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:22.223951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76a9ad-9211-4f55-9453-09d1f5017d02 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:22.224506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 178/333] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Oct 12 12:20:22 2022] GET /identity/v3/groups/9cff2218b9f44d899b7c5d3142fcafd2 => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:22.231282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76a9ad-9211-4f55-9453-09d1f5017d02 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:22.237450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373ee16-a01b-4eef-ac24-7adcc41dd708 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:22.281858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb6416-324d-4c6e-a73a-7a211071c7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:22.282891 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb6416-324d-4c6e-a73a-7a211071c7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:22.283401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb6416-324d-4c6e-a73a-7a211071c7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:22.284563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb6416-324d-4c6e-a73a-7a211071c7a6 None admin] PATH_INFO: `/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:22.285627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb6416-324d-4c6e-a73a-7a211071c7a6 None admin] RBAC: Authorizing `identity:get_group(group_id=8650fd536df34e6ab8cd2bbe7d29aa87)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:22.287755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb6416-324d-4c6e-a73a-7a211071c7a6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:22.289969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 156/334] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Oct 12 12:20:22 2022] GET /identity/v3/groups/8650fd536df34e6ab8cd2bbe7d29aa87 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:22.295706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb6416-324d-4c6e-a73a-7a211071c7a6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:22.310505 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:22.347159 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76a9ad-9211-4f55-9453-09d1f5017d02 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:22.377305 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:22.380619 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9c433-d6be-4448-80ff-2ed471f80fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:22.381211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9c433-d6be-4448-80ff-2ed471f80fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:22.381487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9c433-d6be-4448-80ff-2ed471f80fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:22.381758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9c433-d6be-4448-80ff-2ed471f80fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:22.382508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9c433-d6be-4448-80ff-2ed471f80fb9 None admin] RBAC: Adding query filter params (group.id=9cff2218b9f44d899b7c5d3142fcafd2, role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=5655a305f69b4cb0901b114b25e5c478) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:22.382844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9c433-d6be-4448-80ff-2ed471f80fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:22.384243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9c433-d6be-4448-80ff-2ed471f80fb9 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:22.389960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 179/335] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Oct 12 12:20:22 2022] GET /identity/v3/role_assignments?group.id=9cff2218b9f44d899b7c5d3142fcafd2&scope.project.id=5655a305f69b4cb0901b114b25e5c478&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:22.398851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb6416-324d-4c6e-a73a-7a211071c7a6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:22.435283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba590a9-fd85-47ea-b628-64b918e266c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:22.435809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba590a9-fd85-47ea-b628-64b918e266c2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:22.436086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba590a9-fd85-47ea-b628-64b918e266c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:22.436356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba590a9-fd85-47ea-b628-64b918e266c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:22.437086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba590a9-fd85-47ea-b628-64b918e266c2 None admin] RBAC: Adding query filter params (group.id=8650fd536df34e6ab8cd2bbe7d29aa87, role.id=5c4536b29c2a4ad2a8d05d6e94e6a186, scope.project.id=8b46ee23713a44848ffd7947384566ae) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:22.437392 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba590a9-fd85-47ea-b628-64b918e266c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:22.438909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba590a9-fd85-47ea-b628-64b918e266c2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:22.445486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 157/336] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Oct 12 12:20:22 2022] GET /identity/v3/role_assignments?group.id=8650fd536df34e6ab8cd2bbe7d29aa87&scope.project.id=8b46ee23713a44848ffd7947384566ae&role.id=5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:25.043776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b502f8-926b-4718-aaba-be99643359c2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:25.044818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b502f8-926b-4718-aaba-be99643359c2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:25.045306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b502f8-926b-4718-aaba-be99643359c2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:25.046907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 180/337] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:25.052321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9508-2e0f-4453-b389-f356bb8a2839 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:25.053137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9508-2e0f-4453-b389-f356bb8a2839 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:25.053706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9508-2e0f-4453-b389-f356bb8a2839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:25.088048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6a5d9508-2e0f-4453-b389-f356bb8a2839 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:25.089518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6a5d9508-2e0f-4453-b389-f356bb8a2839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:25.180335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 158/338] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:25.315771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad89ee-72c5-43a6-8290-c8e802fec326 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:25.316395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad89ee-72c5-43a6-8290-c8e802fec326 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:25.316874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad89ee-72c5-43a6-8290-c8e802fec326 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:25.318840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 181/339] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:25.325544 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be4d2-0a14-4a15-9f8e-59410e449ab8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:25.326224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be4d2-0a14-4a15-9f8e-59410e449ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:25.326714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be4d2-0a14-4a15-9f8e-59410e449ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:25.346861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-851be4d2-0a14-4a15-9f8e-59410e449ab8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:25.348688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-851be4d2-0a14-4a15-9f8e-59410e449ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:25.427694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 159/340] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:25.546128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5f9ea-5f80-4fb7-9cbd-7f28b5a3035e None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:25.546787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5f9ea-5f80-4fb7-9cbd-7f28b5a3035e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:25.547267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5f9ea-5f80-4fb7-9cbd-7f28b5a3035e None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:25.548822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 182/341] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:25.559877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf5b5-8e71-4655-b288-08c4e5fd7801 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:25.560435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf5b5-8e71-4655-b288-08c4e5fd7801 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:25.560929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baf5b5-8e71-4655-b288-08c4e5fd7801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:25.566206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5f9ea-5f80-4fb7-9cbd-7f28b5a3035e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:25.572397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:25.599210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d7baf5b5-8e71-4655-b288-08c4e5fd7801 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:25.600417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d7baf5b5-8e71-4655-b288-08c4e5fd7801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:25.700635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 160/342] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:25.751583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5f9ea-5f80-4fb7-9cbd-7f28b5a3035e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:25.763613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baf5b5-8e71-4655-b288-08c4e5fd7801 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:25.817466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:25.830710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:25.832035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:25.832035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:25.832035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:25.849712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:25.853053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:25.883448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:25.884411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:25.885244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:25.885244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:25.885244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:25.885244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 183/343] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:25.893685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:25.947028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baf5b5-8e71-4655-b288-08c4e5fd7801 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:25.989739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:25.995619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:25.996375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:25.996807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:25.997274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.008034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:26.009633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:26.026007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:26.026907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:26.027758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:26.027758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:26.027758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:26.027758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 161/344] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:26.028654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:26.048011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253901f-ae4a-464b-bb1a-f6d3511e07e9 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:26.102448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:26.104098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.104530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.104952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.105762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:26.106254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:26.108872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:26.135294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 184/345] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:26.136267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:26.148256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78084-7e9f-4eca-84df-829043097851 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:26.189970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:26.190711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.191081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.191422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.195461 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:26.202687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:26.204292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:26.218941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:26.219836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:26.220673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:26.220673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:26.220673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:26.220673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 162/346] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:26.220880 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:26.255177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:26.259886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfed22d-cbe8-4e27-921e-0fdc9db1d107 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:26.295701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b789ca-7a3f-49b2-8ec5-ae2b9a97c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:26.296315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b789ca-7a3f-49b2-8ec5-ae2b9a97c590 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.296598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b789ca-7a3f-49b2-8ec5-ae2b9a97c590 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.296873 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b789ca-7a3f-49b2-8ec5-ae2b9a97c590 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.297520 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b789ca-7a3f-49b2-8ec5-ae2b9a97c590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:26.297832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b789ca-7a3f-49b2-8ec5-ae2b9a97c590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:26.299224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b789ca-7a3f-49b2-8ec5-ae2b9a97c590 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:26.310962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 185/347] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:26.312181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd2b73-fbb4-4048-9898-66b5067f8c8f None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.312474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd2b73-fbb4-4048-9898-66b5067f8c8f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.312721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd2b73-fbb4-4048-9898-66b5067f8c8f None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.314077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 186/348] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:26.314610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd2b73-fbb4-4048-9898-66b5067f8c8f None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:26.327671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e8d97-e1ad-458d-94b6-4942eecf8478 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:26.362667 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1758c26-2c92-4e7b-b055-402fc8b59018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:26.363198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1758c26-2c92-4e7b-b055-402fc8b59018 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.363477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1758c26-2c92-4e7b-b055-402fc8b59018 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.363771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1758c26-2c92-4e7b-b055-402fc8b59018 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.364380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1758c26-2c92-4e7b-b055-402fc8b59018 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:26.365784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1758c26-2c92-4e7b-b055-402fc8b59018 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:26.421624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd2b73-fbb4-4048-9898-66b5067f8c8f None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:26.423977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:26.454814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521756ea-1786-4345-bf25-9c14921b1cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:26.455414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521756ea-1786-4345-bf25-9c14921b1cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.455735 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521756ea-1786-4345-bf25-9c14921b1cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.456001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521756ea-1786-4345-bf25-9c14921b1cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.456637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521756ea-1786-4345-bf25-9c14921b1cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:26.456938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521756ea-1786-4345-bf25-9c14921b1cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:26.458324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521756ea-1786-4345-bf25-9c14921b1cc4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:26.478444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 163/349] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:26 2022] POST /identity/v3/users => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:26.478444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1758c26-2c92-4e7b-b055-402fc8b59018 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:26.478726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 187/350] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:26.478726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7e61b-b114-4e7d-ac64-24854ee369bd None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.478726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7e61b-b114-4e7d-ac64-24854ee369bd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.478726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7e61b-b114-4e7d-ac64-24854ee369bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.497472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-12e7e61b-b114-4e7d-ac64-24854ee369bd None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:26.499578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-12e7e61b-b114-4e7d-ac64-24854ee369bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:26.593493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:26.600572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 188/351] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:26.601778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7e61b-b114-4e7d-ac64-24854ee369bd None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:26.613118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1758c26-2c92-4e7b-b055-402fc8b59018 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:26.689472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febeeec-aa84-459a-858e-6b3583eb95c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:26.690151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febeeec-aa84-459a-858e-6b3583eb95c6 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.690438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febeeec-aa84-459a-858e-6b3583eb95c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.690716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febeeec-aa84-459a-858e-6b3583eb95c6 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.691407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febeeec-aa84-459a-858e-6b3583eb95c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:26.692942 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febeeec-aa84-459a-858e-6b3583eb95c6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:26.728951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:26.737369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7e61b-b114-4e7d-ac64-24854ee369bd None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:26.778777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44bf81-2098-4ee5-88a7-686652b9f2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:26.779996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44bf81-2098-4ee5-88a7-686652b9f2da None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.779996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44bf81-2098-4ee5-88a7-686652b9f2da None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.780162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44bf81-2098-4ee5-88a7-686652b9f2da None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.780962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44bf81-2098-4ee5-88a7-686652b9f2da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:26.781191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 164/352] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:26 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:26.786075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44bf81-2098-4ee5-88a7-686652b9f2da None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:26.875193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 189/353] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:26 2022] POST /identity/v3/users => generated 248 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:26.886087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b77bc-c51d-49f8-a368-a8d0755aac21 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.886731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b77bc-c51d-49f8-a368-a8d0755aac21 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.887238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b77bc-c51d-49f8-a368-a8d0755aac21 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.889192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 165/354] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:26.895566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd716b1a-d298-4942-a48b-d159fa77da69 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:26.895876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd716b1a-d298-4942-a48b-d159fa77da69 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:26.896129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd716b1a-d298-4942-a48b-d159fa77da69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:26.913663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b77bc-c51d-49f8-a368-a8d0755aac21 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:26.925112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-fd716b1a-d298-4942-a48b-d159fa77da69 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:26.926447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fd716b1a-d298-4942-a48b-d159fa77da69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:27.046899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 190/355] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:27.077475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b77bc-c51d-49f8-a368-a8d0755aac21 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:27.115122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:27.115745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:27.116069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:27.116342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:27.133738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:27.135152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:27.146564 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:27.154146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:27.155452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:27.156281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:27.156281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:27.156281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:27.156281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 166/356] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:27.163525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd716b1a-d298-4942-a48b-d159fa77da69 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:27.262612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:27.282955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd716b1a-d298-4942-a48b-d159fa77da69 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:27.317303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:27.317987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:27.318697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:27.318975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:27.319634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:27.320822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:27.322854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:27.334982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 191/357] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:27.340974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:27.366150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:27.491874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df145f-3532-4cb0-966d-47988af63997 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:27.527325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28279dc1-6496-4199-ad15-a693de124c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:27.527943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28279dc1-6496-4199-ad15-a693de124c86 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:27.528280 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28279dc1-6496-4199-ad15-a693de124c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:27.528562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28279dc1-6496-4199-ad15-a693de124c86 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:27.529219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28279dc1-6496-4199-ad15-a693de124c86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:27.530689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28279dc1-6496-4199-ad15-a693de124c86 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:27.540373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8668b-d821-4708-ace3-7dd61b7d6ca4 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:27.576861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:27.577800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:27.578335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:27.578830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:27.579736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:27.581265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:27.615534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 167/358] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:27 2022] POST /identity/v3/users => generated 249 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:20:27.639773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.sql.core [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 12:20:27.639773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 12 12:20:27.639773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [parameters: {'id': '8a11670907c446cda77fc482e58a9169', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 12 12:20:27.639773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81490) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result.read() Oct 12 12:20:27.663524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return ref.to_dict() Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:20:27.665185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application yield resource Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise self.value Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application session.commit() Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:20:27.666320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:20:27.667325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:20:27.668174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result.read() Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:20:27.668967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [parameters: {'id': '8a11670907c446cda77fc482e58a9169', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:27.670079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 12 12:20:27.671054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:27.673910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 192/359] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Oct 12 12:20:27 2022] POST /identity/v3/roles => generated 144 bytes in 308 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:27.679755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28279dc1-6496-4199-ad15-a693de124c86 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:27.762206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:27.824480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28279dc1-6496-4199-ad15-a693de124c86 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:27.866184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcf6e4-3e54-43b3-8df8-314b4d19fa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:27.867206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcf6e4-3e54-43b3-8df8-314b4d19fa99 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:27.867688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcf6e4-3e54-43b3-8df8-314b4d19fa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:27.868158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcf6e4-3e54-43b3-8df8-314b4d19fa99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:27.878877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcf6e4-3e54-43b3-8df8-314b4d19fa99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6ffcf6e4-3e54-43b3-8df8-314b4d19fa99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:27.885028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:27.885999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:20:27.886909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:27.889563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 168/360] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:20:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:27.895090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:27.902246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:28.021504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3faebf-d966-4d55-a11d-e78e557cbcc5 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:28.058319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998a283-d4fe-48b8-8032-80d9cd7a7b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:28.058885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998a283-d4fe-48b8-8032-80d9cd7a7b87 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:28.059161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998a283-d4fe-48b8-8032-80d9cd7a7b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:28.059427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998a283-d4fe-48b8-8032-80d9cd7a7b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:28.060092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998a283-d4fe-48b8-8032-80d9cd7a7b87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:28.060389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998a283-d4fe-48b8-8032-80d9cd7a7b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:28.061811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998a283-d4fe-48b8-8032-80d9cd7a7b87 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:28.074006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 193/361] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:29.352874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd94e1-4891-41cb-a4f2-860572a5ff09 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:29.353378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd94e1-4891-41cb-a4f2-860572a5ff09 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:29.353378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd94e1-4891-41cb-a4f2-860572a5ff09 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:29.354926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 169/362] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:29.360294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f799b2-0b18-4673-b5f3-9309f9029f5d None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:29.361114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f799b2-0b18-4673-b5f3-9309f9029f5d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:29.361614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f799b2-0b18-4673-b5f3-9309f9029f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:29.373046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2653-9659-413b-91bc-933a4ebbedd7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:29.373328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2653-9659-413b-91bc-933a4ebbedd7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:29.373606 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2653-9659-413b-91bc-933a4ebbedd7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:29.375143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 170/363] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:29.382121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1aca5-5577-46a1-a333-a02d305944be None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:29.382121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1aca5-5577-46a1-a333-a02d305944be None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:29.382121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1aca5-5577-46a1-a333-a02d305944be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:29.402948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a8f799b2-0b18-4673-b5f3-9309f9029f5d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:29.406069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a8f799b2-0b18-4673-b5f3-9309f9029f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:29.413902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-37b1aca5-5577-46a1-a333-a02d305944be None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:29.415600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-37b1aca5-5577-46a1-a333-a02d305944be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:29.522378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 171/364] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 12 12:20:29.531807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:29.543513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 194/365] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:29.647919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2637e-0112-4185-a904-d715621d721d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:29.648205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2637e-0112-4185-a904-d715621d721d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:29.648952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2637e-0112-4185-a904-d715621d721d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:29.650541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 172/366] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:29.655706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d7442-7277-4418-bd4c-31253a6d6e68 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:29.655991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d7442-7277-4418-bd4c-31253a6d6e68 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:29.656264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d7442-7277-4418-bd4c-31253a6d6e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:29.682868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c67d7442-7277-4418-bd4c-31253a6d6e68 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:29.684056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c67d7442-7277-4418-bd4c-31253a6d6e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:29.760988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 195/367] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:29.867684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2637e-0112-4185-a904-d715621d721d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:29.925986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:29.940028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d7442-7277-4418-bd4c-31253a6d6e68 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:29.995674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2637e-0112-4185-a904-d715621d721d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.047708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0112905-8aca-4319-a76f-f86c628fed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.048346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0112905-8aca-4319-a76f-f86c628fed7b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.048552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d7442-7277-4418-bd4c-31253a6d6e68 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.048661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0112905-8aca-4319-a76f-f86c628fed7b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.048870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0112905-8aca-4319-a76f-f86c628fed7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.076425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:30.082863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0112905-8aca-4319-a76f-f86c628fed7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-a0112905-8aca-4319-a76f-f86c628fed7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:30.090091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:30.091715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:30.092501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 173/368] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:30.093162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a323-5b69-4feb-89cb-4844ede3c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.093162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a323-5b69-4feb-89cb-4844ede3c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.093322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a323-5b69-4feb-89cb-4844ede3c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.093928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a323-5b69-4feb-89cb-4844ede3c3b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.104861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0112905-8aca-4319-a76f-f86c628fed7b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.119827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a323-5b69-4feb-89cb-4844ede3c3b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-8b40a323-5b69-4feb-89cb-4844ede3c3b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:30.124991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:30.125878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:30.126707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:30.134262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 196/369] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:30.137121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a323-5b69-4feb-89cb-4844ede3c3b2 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.257991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a323-5b69-4feb-89cb-4844ede3c3b2 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.287807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0112905-8aca-4319-a76f-f86c628fed7b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.296845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:30.297028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.297918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.298250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.298530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.299209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] RBAC: Adding query filter params (name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:30.299522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:30.300891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:30.310226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 197/370] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:30.312880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.326488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.327199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.327477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.327803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.328589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:30.328892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:30.330269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:30.340151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 174/371] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:30.347192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.380048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:30.438290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761b3d8-d7af-49a7-a945-2596088f7748 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.468765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af18c9-c031-4416-8796-4eeaaf880a3d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.473667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9e0b3-1af4-4606-8f1e-404a1839be40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.478930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9e0b3-1af4-4606-8f1e-404a1839be40 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.478930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9e0b3-1af4-4606-8f1e-404a1839be40 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.478930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9e0b3-1af4-4606-8f1e-404a1839be40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.485836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9e0b3-1af4-4606-8f1e-404a1839be40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-46b9e0b3-1af4-4606-8f1e-404a1839be40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:30.491234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:30.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:30.493584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 198/372] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:30.494287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9e0b3-1af4-4606-8f1e-404a1839be40 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.514173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2856c39-14fd-4a2b-8da9-fc30c09478fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.515082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2856c39-14fd-4a2b-8da9-fc30c09478fa None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.515543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2856c39-14fd-4a2b-8da9-fc30c09478fa None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.515883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2856c39-14fd-4a2b-8da9-fc30c09478fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.527597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2856c39-14fd-4a2b-8da9-fc30c09478fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:30.529071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2856c39-14fd-4a2b-8da9-fc30c09478fa None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-c2856c39-14fd-4a2b-8da9-fc30c09478fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:30.543666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:30.544592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:30.545409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:30.545409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:30.545409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:30.545409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 175/373] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:30.546119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ace0b-e639-49c5-a676-6cad00616834 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.546291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ace0b-e639-49c5-a676-6cad00616834 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.546722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ace0b-e639-49c5-a676-6cad00616834 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.547877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 176/374] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:30.548407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ace0b-e639-49c5-a676-6cad00616834 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.572539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:30.614305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9e0b3-1af4-4606-8f1e-404a1839be40 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.623380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:30.648053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.649256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.649534 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.649812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.658979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ace0b-e639-49c5-a676-6cad00616834 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.660399 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:30.661847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:30.676774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:30.677855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:30.678747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:30.678747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:30.678747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:30.683255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 199/375] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:30.684085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.701685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d652e-99f4-4330-a144-70df98888331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.702322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d652e-99f4-4330-a144-70df98888331 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.702600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d652e-99f4-4330-a144-70df98888331 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.702878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d652e-99f4-4330-a144-70df98888331 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.703540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d652e-99f4-4330-a144-70df98888331 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:30.703839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d652e-99f4-4330-a144-70df98888331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:30.705197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d652e-99f4-4330-a144-70df98888331 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:30.714875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 177/376] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:30.716157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65374e32-e18a-4a30-b4d5-5df3af9b58a6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.716443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65374e32-e18a-4a30-b4d5-5df3af9b58a6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.716713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65374e32-e18a-4a30-b4d5-5df3af9b58a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.739467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-65374e32-e18a-4a30-b4d5-5df3af9b58a6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:30.740888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-65374e32-e18a-4a30-b4d5-5df3af9b58a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:30.798675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169ee5c-235d-4c03-8ceb-4f071e83f9e7 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.818298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 178/377] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:30.818298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65374e32-e18a-4a30-b4d5-5df3af9b58a6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.831669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:30.843952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.859005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.859005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.859005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.859005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:30.859005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:30.859005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:30.859915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 200/378] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:30.860731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:30.885985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:30.931811 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65374e32-e18a-4a30-b4d5-5df3af9b58a6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.963169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2861d7-9558-4055-9c2d-168c2bce0af4 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:30.970732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75413d-0f41-4f55-bc50-cc0c61c2d1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:30.971339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75413d-0f41-4f55-bc50-cc0c61c2d1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.971613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75413d-0f41-4f55-bc50-cc0c61c2d1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.971881 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75413d-0f41-4f55-bc50-cc0c61c2d1e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.972514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75413d-0f41-4f55-bc50-cc0c61c2d1e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:30.972813 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75413d-0f41-4f55-bc50-cc0c61c2d1e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:30.974454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75413d-0f41-4f55-bc50-cc0c61c2d1e6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:30.985400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 179/379] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:30.990890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd8c5a-e08b-4a37-a8c3-7b03b791866b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:30.991374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd8c5a-e08b-4a37-a8c3-7b03b791866b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:30.991496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd8c5a-e08b-4a37-a8c3-7b03b791866b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:30.994069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 180/380] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:30.994069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd8c5a-e08b-4a37-a8c3-7b03b791866b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.000842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.001468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.001744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.002055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.014037 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.015867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:31.031968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:31.032878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:31.033705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:31.033705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:31.033705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:31.033705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 201/381] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:31.033925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.075163 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:31.107269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd8c5a-e08b-4a37-a8c3-7b03b791866b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.116096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:31.140473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678b208-cee5-433b-9a29-43fbf8e4ca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.141138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b208-cee5-433b-9a29-43fbf8e4ca28 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.141417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b208-cee5-433b-9a29-43fbf8e4ca28 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.141684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b208-cee5-433b-9a29-43fbf8e4ca28 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.144689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52724f99-f4a9-488d-9a7a-4ea3d62a8a42 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.149641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678b208-cee5-433b-9a29-43fbf8e4ca28 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.151105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678b208-cee5-433b-9a29-43fbf8e4ca28 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-0678b208-cee5-433b-9a29-43fbf8e4ca28 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:31.164034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:31.165074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 181/382] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Oct 12 12:20:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:31.166410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8353-25d6-41b0-8391-837407b0a7e4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.167161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8353-25d6-41b0-8391-837407b0a7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.167161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c8353-25d6-41b0-8391-837407b0a7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.181399 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.182060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.182349 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.182622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.184620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f44c8353-25d6-41b0-8391-837407b0a7e4 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:31.185799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f44c8353-25d6-41b0-8391-837407b0a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:31.190943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.192389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:31.205162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:31.206194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 202/383] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:31.207083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.254087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 182/384] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:31.259241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8353-25d6-41b0-8391-837407b0a7e4 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.307108 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df776ef-f616-4895-a028-3058eca5ba39 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.323236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:31.338213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:31.343506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.345822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.345822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.345822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.345822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:31.345822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.347275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.363670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 203/385] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:31.363670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.365920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c8353-25d6-41b0-8391-837407b0a7e4 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.405321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746857d-bc56-4f49-8b21-58a97377c6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.405974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746857d-bc56-4f49-8b21-58a97377c6dc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.406250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746857d-bc56-4f49-8b21-58a97377c6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.406520 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746857d-bc56-4f49-8b21-58a97377c6dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.416803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746857d-bc56-4f49-8b21-58a97377c6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-7746857d-bc56-4f49-8b21-58a97377c6dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:31.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:31.422640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:31.423445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 183/386] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:31.424100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746857d-bc56-4f49-8b21-58a97377c6dc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.469316 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca7d89-1f91-479b-a4d9-96a85e32cf36 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.505745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.506371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.506649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.507206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.507656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:31.508482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.509900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.530632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746857d-bc56-4f49-8b21-58a97377c6dc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.546157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 204/387] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:31.551310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.553419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:31.565100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.565711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.568046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.568046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.568046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:31.568046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.568744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.600883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:31.606881 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 184/388] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:31.607706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.654992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daea27-30b2-4bb7-ba9c-3c0f452f9afc None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.688353 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.688975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.689253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.689522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.697344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.698831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.704100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7681-fc64-4524-aa82-bc3d2fa58ca9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:31.709771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:31.710681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:31.711585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 205/389] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:31.712135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.740232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.740837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.741114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.741374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.742592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:31.742903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.744285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.752048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 185/390] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:31.752756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.795988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:31.811141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df063adc-a828-4908-912b-b047029b3c19 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.821063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:31.843306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.843924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.852456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.852456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.852656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8b6c1-b6de-40c5-8646-a815574cc6d7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:31.855312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.856746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:31.871079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:31.872095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:31.872940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:31.872940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:31.872940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:31.872940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 206/391] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:31.872940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.885173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:31.885794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:31.886102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:31.886373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:31.896573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:31.898022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:31.911886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:31.912841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:31.918902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:31.918902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:31.918902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:31.918902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 186/392] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:31.918902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:31.971692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01aed88-62f2-41c7-adad-447cabfe8490 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.004998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.005745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.006403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.006678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.006951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.011583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364ece1-8984-4c6f-b706-884607f258a0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.017032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.018500 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:32.032824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:32.033907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:32.034778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:32.034778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:32.034778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:32.034778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 207/393] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:32.034778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.049231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.049830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.050142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.050414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.051131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:32.051435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.052848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.053786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.072182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 187/394] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Oct 12 12:20:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:32.073016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.136494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f48d3-2998-4f8e-aadc-7b1ecdfb18b3 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.170189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.170730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84e5aa-848d-467b-a8cc-8b393cacb0d0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.170841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.171066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.171331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.181130 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.182597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:32.196645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:32.197714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:32.198943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:32.198943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:32.198943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:32.198943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 208/395] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:32.198943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.206979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.207611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.207893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.208163 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.208799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:32.209109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.210496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.221898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 188/396] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:32.227500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.241544 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.261759 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.295889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76acc-e0d2-4321-9a63-d186b8a9c948 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.323536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422266a-478b-4059-8496-8d19958b6b10 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.328736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.329331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.329610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.329879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.330545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:32.330844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.332199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.344831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 209/397] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:32.345617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.362195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.362799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.363074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.363344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.363974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:32.364276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.365629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.376968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 189/398] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:32.377744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.443694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eea7c7-4929-4630-8410-96c0b66e6087 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.444861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.469436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.473876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6312cb7-c8cd-4805-855a-ff5b1a5d5131 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.478180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.478803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.479085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.479359 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.489237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.490710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:32.505216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:32.506287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:32.507126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:32.507126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:32.507126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:32.508194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.508825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.509103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.509373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.510044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:32.510344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.511779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 210/399] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:32.511888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.512421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.523092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 190/400] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:32.524428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.612404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d1514-247e-4108-b92b-e0ce318f736a None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.623921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad2284-a340-47fb-b291-8fb4a7e8a714 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.645266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953a460-5166-4d10-ac72-53d781d3c430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.645867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953a460-5166-4d10-ac72-53d781d3c430 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.646186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953a460-5166-4d10-ac72-53d781d3c430 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.646458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953a460-5166-4d10-ac72-53d781d3c430 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.647092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953a460-5166-4d10-ac72-53d781d3c430 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.649113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953a460-5166-4d10-ac72-53d781d3c430 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.653090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.657820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.658473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.658753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.659026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.665538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.666984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:32.678423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:32.679403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 191/401] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:32.680290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.750745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 211/402] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:32 2022] POST /identity/v3/users => generated 247 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:32.751492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953a460-5166-4d10-ac72-53d781d3c430 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.785416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a83167-d6e9-4e7d-91e3-2f1d1d6cf975 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.814459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.823332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.823986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.824270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.824546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.832286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.833895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:32.846635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:32.847593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 192/403] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:32.849331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.870173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953a460-5166-4d10-ac72-53d781d3c430 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:32.908850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:32.911287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:32.911911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:32.912192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:32.912465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:32.913186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:32.913492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:32.915137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:32.930304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 212/404] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:32.930477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:32.961618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d41b3-cedd-45f1-bed0-23ad11625cf8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.000124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.000707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.001068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.001263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.009710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.011209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:33.022652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:33.023583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:33.024486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 193/405] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:33.025049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.040890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:33.047581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba569d-2dda-44fe-a333-936deae0354d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.087437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.088044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.088315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.088586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.089280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:33.089575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.091083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.113811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 213/406] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:33.113811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.142038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85106a73-5770-4b7c-8050-23393ecb0ca2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.157508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:33.181421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.182437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.182778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.183056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.183746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:33.184045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.185473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.203821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 194/407] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:33.204640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.230388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff56878-29a6-4dbe-9e8a-ba3931488769 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.267047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.268241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.268943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.269387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.279377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.281027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:33.295250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:33.296118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 214/408] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:33.297049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.317816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da8030-4ce8-4710-939a-10ca34bab47e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.318052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:33.354121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.354819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.355095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.355363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.356076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:33.356377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.357796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.376194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 195/409] 10.222.0.44 () {56 vars in 1139 bytes} [Wed Oct 12 12:20:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:33.376974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.408929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:33.412139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dd2c6-7436-4693-b8a6-16161bf7362c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.449125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69e583-3ea4-4b55-b943-4223060c4686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.450102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69e583-3ea4-4b55-b943-4223060c4686 None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.450379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69e583-3ea4-4b55-b943-4223060c4686 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.450655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69e583-3ea4-4b55-b943-4223060c4686 None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/a0a7230c6da74e6484db3f260b4c58df/roles/7fcd00bd7c614a9587d3161df27b1cff` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.466855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69e583-3ea4-4b55-b943-4223060c4686 None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=a0a7230c6da74e6484db3f260b4c58df, role_id=7fcd00bd7c614a9587d3161df27b1cff)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.468290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d69e583-3ea4-4b55-b943-4223060c4686 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.484558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ed069-98e0-4812-b142-b33e10f790db None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.518453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbffdc5-a153-4234-917f-dcf423ab6dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.518654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbffdc5-a153-4234-917f-dcf423ab6dec None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.518801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbffdc5-a153-4234-917f-dcf423ab6dec None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.519074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbffdc5-a153-4234-917f-dcf423ab6dec None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/70cbcaeb843e470781661f00ec91bffd/roles/7fcd00bd7c614a9587d3161df27b1cff` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.533558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbffdc5-a153-4234-917f-dcf423ab6dec None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=70cbcaeb843e470781661f00ec91bffd, role_id=7fcd00bd7c614a9587d3161df27b1cff)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.535024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbffdc5-a153-4234-917f-dcf423ab6dec None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.543251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 215/410] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:33 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/a0a7230c6da74e6484db3f260b4c58df/roles/7fcd00bd7c614a9587d3161df27b1cff => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:33.544085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69e583-3ea4-4b55-b943-4223060c4686 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.626606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:33.633429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 196/411] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:33 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/70cbcaeb843e470781661f00ec91bffd/roles/7fcd00bd7c614a9587d3161df27b1cff => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:33.635031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbffdc5-a153-4234-917f-dcf423ab6dec None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.684208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69e583-3ea4-4b55-b943-4223060c4686 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.733700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.735577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.738366 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.738646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.739420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:33.739733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.741325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.755239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 216/412] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:33.761319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.800326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:33.826261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbffdc5-a153-4234-917f-dcf423ab6dec None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.878201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.879636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.880380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.880853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.885416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873a0e2-23bc-4be4-b264-f6a3c5d17580 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:33.894386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.896779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:33.915828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:33.919714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:33.920589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:33.921409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:33.921409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:33.921409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:33.923407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 197/413] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:33.934914 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:33.936344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f31e7-f486-4bff-81d5-8f9cbeb40d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:33.937275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f31e7-f486-4bff-81d5-8f9cbeb40d8c None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:33.937753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f31e7-f486-4bff-81d5-8f9cbeb40d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:33.938300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f31e7-f486-4bff-81d5-8f9cbeb40d8c None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/8238fbc3012844ea8b2b5ab13c8ea5d8/roles/7fcd00bd7c614a9587d3161df27b1cff` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:33.957001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f31e7-f486-4bff-81d5-8f9cbeb40d8c None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=8238fbc3012844ea8b2b5ab13c8ea5d8, role_id=7fcd00bd7c614a9587d3161df27b1cff)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:33.958983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f31e7-f486-4bff-81d5-8f9cbeb40d8c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:34.041075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 217/414] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:33 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/8238fbc3012844ea8b2b5ab13c8ea5d8/roles/7fcd00bd7c614a9587d3161df27b1cff => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:20:34.061447 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e3cf5-63f6-4b9e-84e3-978d04982a14 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:34.106317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:34.111268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:34.111899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:34.112179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:34.112451 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:34.113111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:34.113415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:34.114922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:34.135263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 198/415] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:34.141214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f31e7-f486-4bff-81d5-8f9cbeb40d8c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:34.257105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f31e7-f486-4bff-81d5-8f9cbeb40d8c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:34.294255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:34.294887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:34.295164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:34.295435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:34.302336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:34.303799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:34.315652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:34.316504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:34.317367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 218/416] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:34.323142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:34.460747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891b410-f048-4767-b435-136ec5f54acf None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:34.496005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7b634-349d-4fe6-9eff-74a191bfab68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:34.496618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b634-349d-4fe6-9eff-74a191bfab68 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:34.496896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b634-349d-4fe6-9eff-74a191bfab68 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:34.497164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b634-349d-4fe6-9eff-74a191bfab68 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:34.497849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7b634-349d-4fe6-9eff-74a191bfab68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:34.498189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7b634-349d-4fe6-9eff-74a191bfab68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:34.499718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7b634-349d-4fe6-9eff-74a191bfab68 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:34.512124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 199/417] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:34.517717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:34.553760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:34.623057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17a39b-1eb6-4125-9050-92df136b0566 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:34.660311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddceb536-46be-4a01-960c-9329c2f4a7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:34.660970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddceb536-46be-4a01-960c-9329c2f4a7a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:34.661248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddceb536-46be-4a01-960c-9329c2f4a7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:34.661521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddceb536-46be-4a01-960c-9329c2f4a7a6 None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/db04575738b04f2d95803498e19554c4/roles/7fcd00bd7c614a9587d3161df27b1cff` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:34.691543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddceb536-46be-4a01-960c-9329c2f4a7a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=db04575738b04f2d95803498e19554c4, role_id=7fcd00bd7c614a9587d3161df27b1cff)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:34.693029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddceb536-46be-4a01-960c-9329c2f4a7a6 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:34.788821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 219/418] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:34 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/db04575738b04f2d95803498e19554c4/roles/7fcd00bd7c614a9587d3161df27b1cff => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:35.247500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7697fb-259b-4270-97a9-ec953b0ef12b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:35.248496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7697fb-259b-4270-97a9-ec953b0ef12b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:35.249044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7697fb-259b-4270-97a9-ec953b0ef12b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:35.250887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 200/419] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:35.255810 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6115d9-d61d-4927-9f3f-fcceb94d6f2f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:35.256319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6115d9-d61d-4927-9f3f-fcceb94d6f2f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:35.256793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6115d9-d61d-4927-9f3f-fcceb94d6f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:35.287175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9d6115d9-d61d-4927-9f3f-fcceb94d6f2f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:35.288751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9d6115d9-d61d-4927-9f3f-fcceb94d6f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:35.383874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 220/420] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:35.731086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7697fb-259b-4270-97a9-ec953b0ef12b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:35.788184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:35.838545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7697fb-259b-4270-97a9-ec953b0ef12b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:35.874432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d82a6-b2a1-4dc0-9d21-3c65e1debfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:35.875063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d82a6-b2a1-4dc0-9d21-3c65e1debfbd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:35.875348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d82a6-b2a1-4dc0-9d21-3c65e1debfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:35.875625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d82a6-b2a1-4dc0-9d21-3c65e1debfbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:35.886302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d82a6-b2a1-4dc0-9d21-3c65e1debfbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-3b8d82a6-b2a1-4dc0-9d21-3c65e1debfbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:35.891540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:35.892390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:35.893177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 201/421] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:35.899049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6115d9-d61d-4927-9f3f-fcceb94d6f2f None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:35.915325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:36.031675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6115d9-d61d-4927-9f3f-fcceb94d6f2f None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:36.056795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87abf-e683-4c35-b434-9fa805c40190 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.057077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87abf-e683-4c35-b434-9fa805c40190 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.057352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b87abf-e683-4c35-b434-9fa805c40190 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.058895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 202/422] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:20:36.075713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4abbd-00fd-413a-a801-2fdf4a07bde2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.075996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4abbd-00fd-413a-a801-2fdf4a07bde2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.076272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4abbd-00fd-413a-a801-2fdf4a07bde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.102340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-93c4abbd-00fd-413a-a801-2fdf4a07bde2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:36.103537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-93c4abbd-00fd-413a-a801-2fdf4a07bde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:36.107913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:36.108519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.108797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.109066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.109730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:36.110068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:36.111537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:36.126070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 221/423] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:36.133395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:36.185025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 203/424] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:36.231233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:36.254487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42bdfc-eff1-4368-b7c5-a31dd58b62cb None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:36.297376 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a067a97-7057-4195-893d-31c0d7f83095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:36.298028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a067a97-7057-4195-893d-31c0d7f83095 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.298306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a067a97-7057-4195-893d-31c0d7f83095 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.298578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a067a97-7057-4195-893d-31c0d7f83095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.309664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a067a97-7057-4195-893d-31c0d7f83095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:36.311127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a067a97-7057-4195-893d-31c0d7f83095 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-3a067a97-7057-4195-893d-31c0d7f83095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:36.336551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:36.337490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:36.338367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:36.338367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:36.338367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:36.338367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 222/425] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:36.343149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4abbd-00fd-413a-a801-2fdf4a07bde2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:36.406325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:36.465910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4abbd-00fd-413a-a801-2fdf4a07bde2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:36.482078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0760c9a-dd6b-4029-8d57-9c72feca4aa4 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.483206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0760c9a-dd6b-4029-8d57-9c72feca4aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.483206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0760c9a-dd6b-4029-8d57-9c72feca4aa4 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.485509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 223/426] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:20:36.490475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc1860-f3f7-4355-89ad-8eb1407fb89e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.490754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc1860-f3f7-4355-89ad-8eb1407fb89e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.491029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc1860-f3f7-4355-89ad-8eb1407fb89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.522757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-47cc1860-f3f7-4355-89ad-8eb1407fb89e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:36.524281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:36.525119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.525613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.526106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.527008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:36.527507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:36.529073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:36.530513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-47cc1860-f3f7-4355-89ad-8eb1407fb89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:36.554159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 204/427] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:36.559508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:36.653868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 224/428] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:36.655685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc1860-f3f7-4355-89ad-8eb1407fb89e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:36.728722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26159-6e85-40ca-9bc1-81be4038e0aa None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:36.733775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:36.767710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:36.795569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:36.796571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.797049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.797550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.819933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:36.822977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:36.836795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc1860-f3f7-4355-89ad-8eb1407fb89e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:36.843916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:36.844800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:20:36.845675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:36.854987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 205/429] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:36.860663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:36.901787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fad14-6607-4370-bce8-52a187f3dea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:36.903236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fad14-6607-4370-bce8-52a187f3dea3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:36.903740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fad14-6607-4370-bce8-52a187f3dea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:36.904208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fad14-6607-4370-bce8-52a187f3dea3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:36.917255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fad14-6607-4370-bce8-52a187f3dea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-935fad14-6607-4370-bce8-52a187f3dea3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:36.923812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:36.924732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:36.925521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:36.928170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 225/430] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:36.934447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fad14-6607-4370-bce8-52a187f3dea3 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.033819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c75315-f2c0-430e-9d98-e5e16906c395 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.059243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fad14-6607-4370-bce8-52a187f3dea3 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.071043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.072103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.072103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.072408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.073444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:37.074076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:37.075573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:37.079837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:37.094550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 206/431] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Oct 12 12:20:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:37.095323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.098453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b94e3-932e-4844-9b2f-f1604c934a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.099068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b94e3-932e-4844-9b2f-f1604c934a23 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.099346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b94e3-932e-4844-9b2f-f1604c934a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.099696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b94e3-932e-4844-9b2f-f1604c934a23 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.100360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b94e3-932e-4844-9b2f-f1604c934a23 None admin] RBAC: Adding query filter params (name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:37.100665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b94e3-932e-4844-9b2f-f1604c934a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:37.102077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b94e3-932e-4844-9b2f-f1604c934a23 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:37.111441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 226/432] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:37.112928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff08c76-cee0-4a98-bd33-4204faeadd4d None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.113392 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff08c76-cee0-4a98-bd33-4204faeadd4d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.113845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff08c76-cee0-4a98-bd33-4204faeadd4d None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.115246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 227/433] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:37.116296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff08c76-cee0-4a98-bd33-4204faeadd4d None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.124897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:37.212157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a6cf3-d1ec-41c1-890c-86ccaa1c8bd5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.227777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff08c76-cee0-4a98-bd33-4204faeadd4d None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.248356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5e57f-db5b-42a0-b6cc-6e24690dfc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.248981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e57f-db5b-42a0-b6cc-6e24690dfc73 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.249255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e57f-db5b-42a0-b6cc-6e24690dfc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.249514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5e57f-db5b-42a0-b6cc-6e24690dfc73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.261491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5e57f-db5b-42a0-b6cc-6e24690dfc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:37.265431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9d403-c95b-4c98-9990-4c95322e6da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.266079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9d403-c95b-4c98-9990-4c95322e6da2 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.266355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9d403-c95b-4c98-9990-4c95322e6da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-57a5e57f-db5b-42a0-b6cc-6e24690dfc73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:37.266453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:37.267289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:37.268135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:37.269351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9d403-c95b-4c98-9990-4c95322e6da2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.269420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 207/434] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:37.269420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5e57f-db5b-42a0-b6cc-6e24690dfc73 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.281834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9d403-c95b-4c98-9990-4c95322e6da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:37.283299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9d403-c95b-4c98-9990-4c95322e6da2 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-71c9d403-c95b-4c98-9990-4c95322e6da2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:37.300845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:37.301833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:37.302703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:37.302703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:37.302703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:37.302703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 228/435] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:37.303824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bc79c-1c1a-4302-bbb6-acf13162e3ac None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.303824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bc79c-1c1a-4302-bbb6-acf13162e3ac None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.304005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bc79c-1c1a-4302-bbb6-acf13162e3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.324126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c89bc79c-1c1a-4302-bbb6-acf13162e3ac None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:37.325254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c89bc79c-1c1a-4302-bbb6-acf13162e3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:37.359013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:37.378175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:37.399469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5e57f-db5b-42a0-b6cc-6e24690dfc73 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.414095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 229/436] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:37.415023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bc79c-1c1a-4302-bbb6-acf13162e3ac None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.441754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.442605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.443046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.443702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.455558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:37.457068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:37.472867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:37.474068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:37.474903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:37.474903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:37.474903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:37.474903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 208/437] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:37.475119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.537197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bc79c-1c1a-4302-bbb6-acf13162e3ac None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.588881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd12656-475c-422f-a776-fb7d0b7e904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.589700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd12656-475c-422f-a776-fb7d0b7e904b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.590146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd12656-475c-422f-a776-fb7d0b7e904b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.590642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd12656-475c-422f-a776-fb7d0b7e904b None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.591455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd12656-475c-422f-a776-fb7d0b7e904b None admin] RBAC: Adding query filter params (name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:37.591951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd12656-475c-422f-a776-fb7d0b7e904b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:37.593440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd12656-475c-422f-a776-fb7d0b7e904b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:37.608700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 230/438] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:37.608700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c4de-d593-4139-9285-5c39a45dc2b2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.608700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c4de-d593-4139-9285-5c39a45dc2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.608700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305c4de-d593-4139-9285-5c39a45dc2b2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.608700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 231/439] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:37.609466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305c4de-d593-4139-9285-5c39a45dc2b2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.618682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bd37f-db83-435d-a78a-e0d4efae830b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.641357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:37.653481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:37.658019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.658944 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.659454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.659988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.660633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:37.660984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:37.662651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:37.675789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 209/440] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:37.676627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.724769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305c4de-d593-4139-9285-5c39a45dc2b2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.762102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24992a60-af4f-4eec-b472-6c1993f7b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.762705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24992a60-af4f-4eec-b472-6c1993f7b178 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.762980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24992a60-af4f-4eec-b472-6c1993f7b178 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.763250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24992a60-af4f-4eec-b472-6c1993f7b178 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.763875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24992a60-af4f-4eec-b472-6c1993f7b178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:37.764175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24992a60-af4f-4eec-b472-6c1993f7b178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:37.765579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24992a60-af4f-4eec-b472-6c1993f7b178 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:37.778005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 232/441] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:37.779214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bf28a-ee25-4a45-97e6-7e396617c117 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.779493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bf28a-ee25-4a45-97e6-7e396617c117 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.779762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bf28a-ee25-4a45-97e6-7e396617c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.789413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb8be1-76ab-40ac-bea9-0991b0bd9c1e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.797731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-021bf28a-ee25-4a45-97e6-7e396617c117 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:37.798950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-021bf28a-ee25-4a45-97e6-7e396617c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:37.822989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.823608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.823887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.824158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:37.833840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:37.835306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:37.849644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:37.850566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:37.852021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:37.852021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:37.852021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:37.852021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 210/442] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:37.852622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.865724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 233/443] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:37.866582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bf28a-ee25-4a45-97e6-7e396617c117 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:37.896521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:37.900171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:37.955208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bab2d-4a94-4baf-b2aa-31d172197611 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.985700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bf28a-ee25-4a45-97e6-7e396617c117 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:37.992679 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f12fc-8042-4a5c-ab93-d8a4dcc5aeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:37.993309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f12fc-8042-4a5c-ab93-d8a4dcc5aeb7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:37.993589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f12fc-8042-4a5c-ab93-d8a4dcc5aeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:37.993863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f12fc-8042-4a5c-ab93-d8a4dcc5aeb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.005377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f12fc-8042-4a5c-ab93-d8a4dcc5aeb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f83f12fc-8042-4a5c-ab93-d8a4dcc5aeb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:38.010689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:38.011548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:38.012355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 211/444] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:38.013609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f12fc-8042-4a5c-ab93-d8a4dcc5aeb7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.026267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.026891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.027169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.027436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.034679 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.036122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:38.048059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:38.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:38.049888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:38.049888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:38.049888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:38.049888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:38.049888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:38.049888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:38.049888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:38.049888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:38.051529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 234/445] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:38.052422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.129113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f12fc-8042-4a5c-ab93-d8a4dcc5aeb7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.132430 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:38.132973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:38.156469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f59c3b-e382-427c-9192-d11066ab4427 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.162938 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.164722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.164722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.164722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.171871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.173311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:38.184878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:38.191196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 212/446] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:38.192081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.192698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.192698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.192698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.192698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.193111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:38.193111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.194064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.206815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 235/447] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:38.207614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.287553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0b6e9-d531-4967-a328-a6d354d3b9d7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.305849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef75850-f2f6-4bf7-b648-6500675d6a4c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.320073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.320689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.320966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.321239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.321895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:38.322252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.323612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.331384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 213/448] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:38.332076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.339768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.340375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.340712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.340989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.341720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:38.342067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.343470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.345068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:38.356501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 236/449] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:38.357281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.363489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:38.436121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da799ab-65f7-40d2-aac5-f8050f013504 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.455572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7b98c-8d3a-44dd-ae8b-5d736cf11715 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.469941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388968cb-8426-482e-9bb3-1f321eba7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.470574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388968cb-8426-482e-9bb3-1f321eba7e0f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.470849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388968cb-8426-482e-9bb3-1f321eba7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.471119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388968cb-8426-482e-9bb3-1f321eba7e0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.481079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388968cb-8426-482e-9bb3-1f321eba7e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-388968cb-8426-482e-9bb3-1f321eba7e0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:38.486270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:38.487223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:38.488000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 214/450] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:38.488636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388968cb-8426-482e-9bb3-1f321eba7e0f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.490696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.491299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.491578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.491848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.492596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:38.492901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.494350 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.511683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 237/451] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:38.512486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.563062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:38.571949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:38.587213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388968cb-8426-482e-9bb3-1f321eba7e0f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.609728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ae8fd-c7ab-4fa1-a172-72402a35da31 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.619158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.619786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.620071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.620346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.628667 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.630149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:38.641682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:38.642771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 215/452] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:38.643983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.644601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.644601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.644601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.644792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.657866 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.659338 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:38.673023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:38.674143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:38.674986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:38.674986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:38.674986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:38.674986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 238/453] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:38.675202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.745666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2e2eb-b6a0-46a4-a3d1-dd599d5231bf None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.775184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827abdc8-7f3e-4096-a675-328a060af148 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.777450 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:38.778373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe19cc4-4e3f-4475-8329-c1570ba31e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.779010 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe19cc4-4e3f-4475-8329-c1570ba31e51 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.779281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe19cc4-4e3f-4475-8329-c1570ba31e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.779549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe19cc4-4e3f-4475-8329-c1570ba31e51 None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/36af83e8ee4048d78d2db8784f6347df/roles/7fcd00bd7c614a9587d3161df27b1cff` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.793747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe19cc4-4e3f-4475-8329-c1570ba31e51 None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=36af83e8ee4048d78d2db8784f6347df, role_id=7fcd00bd7c614a9587d3161df27b1cff)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.795213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe19cc4-4e3f-4475-8329-c1570ba31e51 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.798087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:38.808853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.809495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.809770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.810127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.810780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] RBAC: Adding query filter params (name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:38.811081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:38.812402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:38.820716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 239/454] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:38.821539 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.862698 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 216/455] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:38 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/36af83e8ee4048d78d2db8784f6347df/roles/7fcd00bd7c614a9587d3161df27b1cff => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:38.863684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe19cc4-4e3f-4475-8329-c1570ba31e51 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:38.930948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baf94f-f495-49d0-863f-ed3a433d4732 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:38.967200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642c2bf-d6b6-473b-a229-cbbd40e3818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:38.967821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c2bf-d6b6-473b-a229-cbbd40e3818c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:38.968102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c2bf-d6b6-473b-a229-cbbd40e3818c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:38.968372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642c2bf-d6b6-473b-a229-cbbd40e3818c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:38.977600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe19cc4-4e3f-4475-8329-c1570ba31e51 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.007608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.019619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.021568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.021568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.021568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.022375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:39.022752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.024283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.043618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 217/456] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:39.044537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.046481 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642c2bf-d6b6-473b-a229-cbbd40e3818c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.047930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642c2bf-d6b6-473b-a229-cbbd40e3818c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.107708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.157609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 240/457] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:39.157609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642c2bf-d6b6-473b-a229-cbbd40e3818c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.164690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50037abd-b587-40e0-9d34-3ce858754deb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.209488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.210130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.210412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.210708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.211888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:39.212215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.214141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.224594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.227556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 218/458] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:39.228223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.266825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642c2bf-d6b6-473b-a229-cbbd40e3818c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.305417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.306090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.306372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.307046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.319646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.321234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.331655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd46f3d-0306-4f44-8b10-e0e6b2665dc3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:39.335811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:39.336725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:39.337636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:39.337636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:39.337636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:39.337636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 241/459] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:39.338334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.351102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.378323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec1ed9-558d-40b2-bbec-fd6c2c4d6cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.378978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec1ed9-558d-40b2-bbec-fd6c2c4d6cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.379231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec1ed9-558d-40b2-bbec-fd6c2c4d6cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.379882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec1ed9-558d-40b2-bbec-fd6c2c4d6cc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.444033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57a085-eabf-43fa-b51b-c7f9c892bcce None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.458418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec1ed9-558d-40b2-bbec-fd6c2c4d6cc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.459874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec1ed9-558d-40b2-bbec-fd6c2c4d6cc1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.461174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.478775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.479399 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.479680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.479957 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.490499 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.491947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:39.507062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:39.508005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:39.508842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:39.508842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:39.508842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:39.508842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 242/460] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:39.509059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.575606 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 219/461] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:39.576524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec1ed9-558d-40b2-bbec-fd6c2c4d6cc1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.596371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.618797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba7656-d7a3-4543-ac58-6a2c775d361c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.652018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.652652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.652932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.653207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.662941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.665798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:39.677917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:39.678803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:39.680220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 243/462] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:39.680767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.690159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec1ed9-558d-40b2-bbec-fd6c2c4d6cc1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.715980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.726859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.727466 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.727902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.728131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.728777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:39.729084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.730512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.743526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 220/463] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:39.744402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.793537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60599ab-0a36-469d-ba4f-5b50acd46389 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.828482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.830214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.831210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.831496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.831770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.832858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:39.833313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.834723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.846248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 244/464] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:39.848534 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.859800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73957c28-1943-45c6-aef7-4bdf5e1c268e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.895787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.896502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.896871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.897154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.909956 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.911993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:39.926790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:39.927687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:39.928526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:39.928526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:39.928526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:39.928526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 221/465] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:39.928760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:39.947744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:39.957885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc87d12-8815-4b00-8d7f-6d02c71e7a22 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:39.995863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:39.996584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:39.996936 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:39.997283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:39.998106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:39.998488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:39.999974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.024047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 245/466] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:40.024047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.042505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099b98e-808c-46f5-af70-0f2c33793ea3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.076646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:40.100833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.100833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.100833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.100833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.101228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.101228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:40.112485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:40.113407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 222/467] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:40.114677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.135629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe972a24-1ed1-47ee-9f42-ee65a72cfbdb None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.173590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.174276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.174558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.174830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.181600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.183073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:40.194506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:40.195892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:40.196729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 246/468] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:40.197191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.211867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:40.228269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bc1df-5d4e-4db5-8356-6acec63856d0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.264684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.265283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.265566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.265838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.266534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:40.266835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.268190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.279955 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 223/469] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:40.280759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.305786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8160c6-179d-4ec4-bc39-c4792a73f5e8 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.328800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:40.343013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197535-ae96-4a0b-b864-47b9aac9c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.343608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197535-ae96-4a0b-b864-47b9aac9c489 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.343893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197535-ae96-4a0b-b864-47b9aac9c489 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.344167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197535-ae96-4a0b-b864-47b9aac9c489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.387791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bce7f-9d82-4117-acea-b914b8c34274 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.417189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17197535-ae96-4a0b-b864-47b9aac9c489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.418904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17197535-ae96-4a0b-b864-47b9aac9c489 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.425172 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.425779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.426092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.426369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.427097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:40.427411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.428819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.431033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:40.446937 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 224/470] 10.222.0.44 () {56 vars in 1139 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:40.447760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.529514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 247/471] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:40.530694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197535-ae96-4a0b-b864-47b9aac9c489 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.557156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea70bb-2083-4a74-95bf-c6b7884b0fb8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.559515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:40.593237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.594150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.594435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.594706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.595395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:40.595703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.597108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.615528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 225/472] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:40.615528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.639712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197535-ae96-4a0b-b864-47b9aac9c489 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.649754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:40.674730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.675358 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.675701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.675975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.682969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.685251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:40.697726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:40.698622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:40.699449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 248/473] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:40.699938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.729232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31d9d6-fb41-444f-89a7-307b14192621 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.764114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e71f-8017-4747-8603-cab6db8001f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.764713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e71f-8017-4747-8603-cab6db8001f3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.764998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e71f-8017-4747-8603-cab6db8001f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.765275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e71f-8017-4747-8603-cab6db8001f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.797611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:40.807484 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364d6dd-4f00-43ac-8572-856110a79dd8 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:40.836836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e71f-8017-4747-8603-cab6db8001f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.838351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e71f-8017-4747-8603-cab6db8001f3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.842981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:40.843818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:40.844268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:40.844708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:40.855905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:40.857519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:40.872078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:40.872992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:40.874274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:40.874274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:40.874274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:40.874274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 249/474] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:40.874274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.900962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:40.953852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 226/475] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:40.954851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e71f-8017-4747-8603-cab6db8001f3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:40.984189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9842a23-7c2b-4ccc-9985-6a913581f3b0 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.020102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.020718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.021000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.021565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.022324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] RBAC: Adding query filter params (role.id=7fcd00bd7c614a9587d3161df27b1cff, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=70cbcaeb843e470781661f00ec91bffd) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:41.022640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.024111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.026236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:41.030152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 250/476] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/role_assignments?user.id=70cbcaeb843e470781661f00ec91bffd&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7fcd00bd7c614a9587d3161df27b1cff => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:41.030791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.085006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e71f-8017-4747-8603-cab6db8001f3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.124604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.125229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.125507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.125780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.126540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:41.126845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.128280 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.130604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:41.147276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 227/477] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:41.147276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.153321 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d240c16-eb18-4f25-9cad-63f6ff552ccc None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.194364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95b83c-39cf-4d16-a3fd-ec68d34e9fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.194978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95b83c-39cf-4d16-a3fd-ec68d34e9fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.195910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95b83c-39cf-4d16-a3fd-ec68d34e9fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.196188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95b83c-39cf-4d16-a3fd-ec68d34e9fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.196827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95b83c-39cf-4d16-a3fd-ec68d34e9fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:41.197127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95b83c-39cf-4d16-a3fd-ec68d34e9fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.198699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95b83c-39cf-4d16-a3fd-ec68d34e9fa3 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.213200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 251/478] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:41.215309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d8262-440e-42b9-87ff-b267d216b166 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.215590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d8262-440e-42b9-87ff-b267d216b166 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.215861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d8262-440e-42b9-87ff-b267d216b166 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.217993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 252/479] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:41.217993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d8262-440e-42b9-87ff-b267d216b166 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.267357 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe703c-e284-4be2-af1f-1f952f715921 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.292250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:41.302613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.303238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.303511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.307560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.315479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.317099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:41.332885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:41.333826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:41.339559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:41.339559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:41.339559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:41.339876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d8262-440e-42b9-87ff-b267d216b166 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.339961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 228/480] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:41.339961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.374641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab701b0f-9d4c-4ed4-9f5c-69574ae7c131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.375246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab701b0f-9d4c-4ed4-9f5c-69574ae7c131 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.375525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab701b0f-9d4c-4ed4-9f5c-69574ae7c131 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.375801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab701b0f-9d4c-4ed4-9f5c-69574ae7c131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.376524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab701b0f-9d4c-4ed4-9f5c-69574ae7c131 None admin] RBAC: Adding query filter params (role.id=7fcd00bd7c614a9587d3161df27b1cff, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=a0a7230c6da74e6484db3f260b4c58df) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:41.376834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab701b0f-9d4c-4ed4-9f5c-69574ae7c131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.378325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab701b0f-9d4c-4ed4-9f5c-69574ae7c131 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.384374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 253/481] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/role_assignments?user.id=a0a7230c6da74e6484db3f260b4c58df&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7fcd00bd7c614a9587d3161df27b1cff => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:41.385579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8760fa8-3cce-483d-aede-b5a6dec9e14e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.385869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8760fa8-3cce-483d-aede-b5a6dec9e14e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.386171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8760fa8-3cce-483d-aede-b5a6dec9e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.416310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b8760fa8-3cce-483d-aede-b5a6dec9e14e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:41.417779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b8760fa8-3cce-483d-aede-b5a6dec9e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:41.421389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:41.476127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ef566-cb9d-4516-b9a5-b0bab917e452 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.512148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 254/482] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:41.513469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8760fa8-3cce-483d-aede-b5a6dec9e14e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.524572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.525227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.525510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.525786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.534215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.535946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:41.550461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:41.551367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:41.552270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:41.552270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:41.552270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:41.552270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:41.552270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:41.552270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:41.552270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:41.552270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:41.557063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 229/483] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:41.559187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.601986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:41.645007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8760fa8-3cce-483d-aede-b5a6dec9e14e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.687194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3623d6-d4b4-49cd-a4a5-8d26651e3bdf None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.690730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.691549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.691999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.692441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.693271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:41.693744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.695451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.714275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:41.718706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 255/484] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:41.725135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.760257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.760924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.761143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.761414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.762128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:41.762434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.764172 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.783557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 230/485] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:41.785510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.898898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:41.903911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc6e95-6f5b-4cfb-a7db-8d893d6860a4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.926189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236fda-2172-4a34-b5d6-0883829db42f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:41.948060 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.948870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.949159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.949435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.950374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] RBAC: Adding query filter params (role.id=7fcd00bd7c614a9587d3161df27b1cff, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=8238fbc3012844ea8b2b5ab13c8ea5d8) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:41.950694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.952823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.962507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:41.963215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:41.963557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:41.963894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:41.966039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 231/486] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/role_assignments?user.id=8238fbc3012844ea8b2b5ab13c8ea5d8&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7fcd00bd7c614a9587d3161df27b1cff => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:41.967000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:41.976013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:41.978199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:41.990992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:41.991975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:41.992812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 256/487] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:42.007025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:42.141358 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:42.163727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258fad0-2f27-4d98-a74d-b018f76e482e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:42.217531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4176f43-199b-4541-bfa9-c7fbe1a88485 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:42.234881 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20716d38-6713-49fb-b864-95368fa55165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:42.234881 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20716d38-6713-49fb-b864-95368fa55165 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:42.235172 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20716d38-6713-49fb-b864-95368fa55165 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:42.235245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20716d38-6713-49fb-b864-95368fa55165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:42.251301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20716d38-6713-49fb-b864-95368fa55165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-20716d38-6713-49fb-b864-95368fa55165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:42.257442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:42.258893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:42.259678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 232/488] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:42.264793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:42.265667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:42.265968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20716d38-6713-49fb-b864-95368fa55165 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:42.266384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:42.266868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:42.267762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:42.268280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:42.270158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:42.287618 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 257/489] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:42.294075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:42.296925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:42.444276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20716d38-6713-49fb-b864-95368fa55165 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:42.488476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df8302-39af-487f-bc9c-406537fe8c4d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:42.511647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:42.512261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:42.512544 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:42.512819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:42.513487 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:42.513795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:42.515200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:42.522792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:42.536747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 233/490] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:42.541560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:42.557655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:42.558319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:42.558602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:42.558872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:42.559598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] RBAC: Adding query filter params (role.id=7fcd00bd7c614a9587d3161df27b1cff, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=db04575738b04f2d95803498e19554c4) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:42.559910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:42.561342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:42.577596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 258/491] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:42 2022] GET /identity/v3/role_assignments?user.id=db04575738b04f2d95803498e19554c4&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7fcd00bd7c614a9587d3161df27b1cff => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:42.656211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:42.686373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c055f-11d4-4261-bcaa-37d82a68f9d5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:42.741799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:42.743344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:42.743847 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:42.744331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:42.760295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:42.762273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:42.786396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:42.787512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:42.788364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:42.788364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:42.788364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:42.788364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 234/492] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:42.793214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:42.906800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8fdab-3711-46c2-8263-5c5a6dd9adfd None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:42.948574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:42.949531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:42.950061 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:42.950547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:42.951437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:42.951989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:42.953536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:42.966737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 259/493] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:42.972594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:43.114512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56810b0a-2a88-4b54-a176-e901032bee5a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:43.154630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:43.155255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:43.155536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:43.155806 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:43.157736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:43.168570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:43.170032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:43.181762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:43.191456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:43.192456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 235/494] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:43.193013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:43.277360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:43.353062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cc414-1d35-453f-a12e-ac31521f7e8c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:43.406779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6c1e0-4e24-4628-aebd-0acdf34de26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:43.407425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6c1e0-4e24-4628-aebd-0acdf34de26e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:43.407701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6c1e0-4e24-4628-aebd-0acdf34de26e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:43.407968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6c1e0-4e24-4628-aebd-0acdf34de26e None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:43.408727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6c1e0-4e24-4628-aebd-0acdf34de26e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:43.409032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6c1e0-4e24-4628-aebd-0acdf34de26e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:43.410524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6c1e0-4e24-4628-aebd-0acdf34de26e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:43.438129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 260/495] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Oct 12 12:20:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:43.443660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:43.514499 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ddb1-d886-49a4-a929-1183727c614d None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:43.514851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ddb1-d886-49a4-a929-1183727c614d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:43.515048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ddb1-d886-49a4-a929-1183727c614d None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:43.516921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 261/496] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:20:43.525961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720a5fd-f9a3-434e-8d07-bdedc8b63168 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:43.525961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720a5fd-f9a3-434e-8d07-bdedc8b63168 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:43.525961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720a5fd-f9a3-434e-8d07-bdedc8b63168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:43.555040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d720a5fd-f9a3-434e-8d07-bdedc8b63168 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:43.556238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d720a5fd-f9a3-434e-8d07-bdedc8b63168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:43.590888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109ea7c-97be-4b4f-8162-d5b83beeeb91 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:43.629756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6661-1f86-466f-81bf-044df289211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:43.630297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6661-1f86-466f-81bf-044df289211a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:43.630575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6661-1f86-466f-81bf-044df289211a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:43.630843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6661-1f86-466f-81bf-044df289211a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:43.697970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 262/497] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:20:43.728721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:43.737456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef6661-1f86-466f-81bf-044df289211a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:43.743084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef6661-1f86-466f-81bf-044df289211a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:43.851479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f76a6-8c5a-413b-9c62-e96d5398fdef None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:43.851828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f76a6-8c5a-413b-9c62-e96d5398fdef None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:43.852023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f76a6-8c5a-413b-9c62-e96d5398fdef None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:43.853358 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 263/498] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:20:43.858351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89990f-ff7d-4c0d-843d-2667dd440c51 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:43.858617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 236/499] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:20:43.858724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89990f-ff7d-4c0d-843d-2667dd440c51 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:43.859017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89990f-ff7d-4c0d-843d-2667dd440c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:43.867156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6661-1f86-466f-81bf-044df289211a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:43.889158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2d89990f-ff7d-4c0d-843d-2667dd440c51 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:43.891366 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2d89990f-ff7d-4c0d-843d-2667dd440c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:43.934727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:44.003047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6661-1f86-466f-81bf-044df289211a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.004743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:44.026047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 264/500] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:44.026937 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89990f-ff7d-4c0d-843d-2667dd440c51 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.050609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58564a94-8532-43d5-972f-9830e045694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.051239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58564a94-8532-43d5-972f-9830e045694a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.051513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58564a94-8532-43d5-972f-9830e045694a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.051781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58564a94-8532-43d5-972f-9830e045694a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.077969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58564a94-8532-43d5-972f-9830e045694a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:44.077969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58564a94-8532-43d5-972f-9830e045694a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-58564a94-8532-43d5-972f-9830e045694a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:44.101986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:44.102951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:44.103895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:44.103895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:44.103895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:44.111295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 237/501] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:44.112251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58564a94-8532-43d5-972f-9830e045694a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.222092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89990f-ff7d-4c0d-843d-2667dd440c51 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.281493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58564a94-8532-43d5-972f-9830e045694a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.303502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.304992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.305476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.305967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.323279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.323874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.324149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.324417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.325058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:44.325352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:44.326807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:44.329887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:44.329887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:44.339277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 238/502] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:44.343412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:44.347725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:44.349340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:44.350256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:44.350256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:44.350256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:44.354206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:44.357850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 265/503] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:44.358882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.375522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:44.465965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a93bab-ed1d-4750-b0f5-999c0ccbfaeb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.480022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f108b09-218f-4016-968b-2540aa55316f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.502203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfc22f-c7ce-4bc9-b97f-eeb2fc65ee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.502912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfc22f-c7ce-4bc9-b97f-eeb2fc65ee74 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.503195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfc22f-c7ce-4bc9-b97f-eeb2fc65ee74 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.503470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfc22f-c7ce-4bc9-b97f-eeb2fc65ee74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.516734 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfc22f-c7ce-4bc9-b97f-eeb2fc65ee74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-15dfc22f-c7ce-4bc9-b97f-eeb2fc65ee74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:44.521961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:44.522971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:44.524460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 239/504] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:44.525174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfc22f-c7ce-4bc9-b97f-eeb2fc65ee74 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.530670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.530670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.530670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.530670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.541569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:44.543679 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:44.556786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:44.557694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:44.558968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:44.558968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:44.558968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:44.558968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:44.558968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:44.558968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:44.558968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:44.558968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:44.561187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 266/505] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:44.562208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.599892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:44.634870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfc22f-c7ce-4bc9-b97f-eeb2fc65ee74 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.641971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:44.672467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da79e52-b0b8-4627-b674-bff9c29bb30c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.672627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.673067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.673345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.673616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.674275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:44.674572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:44.675934 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:44.691497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 240/506] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:44.691497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.718274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792dcd9-28ca-473a-9ecf-aed92527bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.718883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792dcd9-28ca-473a-9ecf-aed92527bf10 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.719171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792dcd9-28ca-473a-9ecf-aed92527bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.719442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792dcd9-28ca-473a-9ecf-aed92527bf10 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.720106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792dcd9-28ca-473a-9ecf-aed92527bf10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:44.720416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792dcd9-28ca-473a-9ecf-aed92527bf10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:44.721801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792dcd9-28ca-473a-9ecf-aed92527bf10 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:44.734720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 267/507] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:44.734720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cad17e-8350-4dc1-bdf5-063a70e09ada None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.734720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cad17e-8350-4dc1-bdf5-063a70e09ada None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.734720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cad17e-8350-4dc1-bdf5-063a70e09ada None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.734720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 268/508] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:44.734720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cad17e-8350-4dc1-bdf5-063a70e09ada None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.798364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1f4ae-d47b-47a9-b710-c7821093d5d9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.827038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:44.837675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.838569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.838849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.853604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.853604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:44.853604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:44.853604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:44.853887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cad17e-8350-4dc1-bdf5-063a70e09ada None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.858084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 241/509] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:44.859027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:44.885432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090f15-9c74-4528-b659-e43c17629a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:44.886389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090f15-9c74-4528-b659-e43c17629a81 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.886874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090f15-9c74-4528-b659-e43c17629a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.887339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090f15-9c74-4528-b659-e43c17629a81 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:44.888275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090f15-9c74-4528-b659-e43c17629a81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:44.889987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090f15-9c74-4528-b659-e43c17629a81 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:44.898643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:44.958275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93746df-82e9-41b0-b916-16ba19dbf212 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:44.977239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 269/510] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:44 2022] POST /identity/v3/users => generated 254 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:44.979044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e9ec4-4958-4372-805a-8eea94fffeb7 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:44.979324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e9ec4-4958-4372-805a-8eea94fffeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:44.979606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e9ec4-4958-4372-805a-8eea94fffeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.004575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-0a6e9ec4-4958-4372-805a-8eea94fffeb7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:45.005388 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:45.005925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0a6e9ec4-4958-4372-805a-8eea94fffeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:45.006483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.006970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.007439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.021057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:45.023018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:45.040473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:45.041499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:45.047236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:45.047236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:45.047236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:45.047236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 242/511] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:45.047236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:45.102015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:45.114740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 270/512] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:45.115512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e9ec4-4958-4372-805a-8eea94fffeb7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:45.181323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0624727-9744-4f18-957c-0757d9c24895 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:45.249240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:45.250711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.251257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.251751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.252942 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] RBAC: Adding query filter params (role.id=7fcd00bd7c614a9587d3161df27b1cff, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=36af83e8ee4048d78d2db8784f6347df) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:45.253716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:45.255889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:45.264497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 243/513] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:45 2022] GET /identity/v3/role_assignments?user.id=36af83e8ee4048d78d2db8784f6347df&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7fcd00bd7c614a9587d3161df27b1cff => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:45.283986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e9ec4-4958-4372-805a-8eea94fffeb7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:45.310596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:45.330646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:45.331660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.334708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.334708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.334708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:45.334708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:45.335774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:45.350344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 271/514] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:45.357248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:45.435039 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:45.483691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1ca2c-21f3-4117-a6d6-83d763af49e6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:45.485763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:45.525393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120eae9e-19f3-46e8-9eca-6e28c3748416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:45.526026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120eae9e-19f3-46e8-9eca-6e28c3748416 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.526265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120eae9e-19f3-46e8-9eca-6e28c3748416 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.526550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120eae9e-19f3-46e8-9eca-6e28c3748416 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.536650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120eae9e-19f3-46e8-9eca-6e28c3748416 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:45.538147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120eae9e-19f3-46e8-9eca-6e28c3748416 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-120eae9e-19f3-46e8-9eca-6e28c3748416 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:45.550411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:45.551550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 244/515] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Oct 12 12:20:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:45.552878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168492b-6a4a-42cc-8500-b576e6c70e95 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.553641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168492b-6a4a-42cc-8500-b576e6c70e95 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.553641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168492b-6a4a-42cc-8500-b576e6c70e95 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.554835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 245/516] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:45.560933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2e765-efa3-45ef-ad21-ca69bfba6844 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:45.561073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168492b-6a4a-42cc-8500-b576e6c70e95 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:45.609678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8cb80-2c71-405a-b285-ffc4baacd7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:45.611277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8cb80-2c71-405a-b285-ffc4baacd7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.611567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8cb80-2c71-405a-b285-ffc4baacd7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.611846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8cb80-2c71-405a-b285-ffc4baacd7b4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.612436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8cb80-2c71-405a-b285-ffc4baacd7b4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:45.613817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8cb80-2c71-405a-b285-ffc4baacd7b4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-7be8cb80-2c71-405a-b285-ffc4baacd7b4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:45.628863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:45.629894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:45.630792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:45.635236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 272/517] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Oct 12 12:20:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:45.636948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e47f-caf6-43d3-bcf2-c9c7a7faade2 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.637238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e47f-caf6-43d3-bcf2-c9c7a7faade2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.637518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e47f-caf6-43d3-bcf2-c9c7a7faade2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.662539 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-1281e47f-caf6-43d3-bcf2-c9c7a7faade2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:45.666434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1281e47f-caf6-43d3-bcf2-c9c7a7faade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:45.691267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168492b-6a4a-42cc-8500-b576e6c70e95 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:45.709769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:45.731092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:45.731704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.731982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.732259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.733000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:45.733296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:45.734759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:45.746321 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 273/518] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:45.747631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281e47f-caf6-43d3-bcf2-c9c7a7faade2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:45.753420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 246/519] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Oct 12 12:20:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:45.759270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:45.786355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:45.895932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281e47f-caf6-43d3-bcf2-c9c7a7faade2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:45.933719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:45.934360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.934674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.934948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:45.935622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:45.935929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:45.936039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d734ce-db1f-4e96-b24e-6c5609c95b78 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:45.937283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:45.942501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 274/520] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Oct 12 12:20:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:45.950953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:45.997973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:45.998849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:45.999294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:45.999704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:46.022024 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:46.022220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:46.024144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:46.049910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:46.051101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:46.052297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:46.052297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:46.052297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:46.052297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 247/521] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:46.058125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:46.148850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:46.151596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2ed53-ff95-4912-8dee-89adcc6742c2 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:46.180511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30222b1b-26ea-473d-b734-78061cbbf914 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:46.201049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:46.202088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:46.202374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:46.202651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:46.203298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:46.203604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:46.204984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:46.214599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 275/522] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Oct 12 12:20:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:46.216023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:46.224632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:46.225537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:46.226003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:46.226489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:46.227284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:46.227741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:46.229364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:46.245597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 248/523] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:46.255116 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:46.370831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabbde-7b53-4f19-8096-2243b4b5956d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:46.374282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:46.428554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0400fd8-9ece-469b-b0f9-cdbedf74dcaa None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:46.456041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:46.456961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:46.457477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:46.457892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:46.458692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:46.460204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:46.463243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:46.477820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:46.479069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:46.480011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:46.481436 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:46.482263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:46.482697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:46.483110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:46.486369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 276/524] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:46 2022] GET /identity/v3/services/network => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:46.492720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:46.492885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:46.494578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:46.508155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:46.509203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:46.510258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 249/525] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:46.519691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:46.662306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db633f9c-acdd-4c74-b540-79bc9912aaa1 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:46.701286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10130b4-fbaa-465e-be17-c5627dd1626f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:46.733835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:46.734468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:46.734746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:46.735017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:46.735645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] RBAC: Adding query filter params (name=network) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:46.735951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:46.737305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:46.745448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 277/526] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:20:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:46.750867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:46.766890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:46.767512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:46.767793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:46.768070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:46.768769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:46.769076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:46.770677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:46.793881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:46.800141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 250/527] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:46.806164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:46.896807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:46.902045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145ee77-6110-4c5f-89ce-7f0783a3a28d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:46.966613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef66a5-e7a3-4fca-a9d3-7e954b97ffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:46.967138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef66a5-e7a3-4fca-a9d3-7e954b97ffa7 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:46.967413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef66a5-e7a3-4fca-a9d3-7e954b97ffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:46.967680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef66a5-e7a3-4fca-a9d3-7e954b97ffa7 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:46.968301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef66a5-e7a3-4fca-a9d3-7e954b97ffa7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:46.968598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef66a5-e7a3-4fca-a9d3-7e954b97ffa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:46.970030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef66a5-e7a3-4fca-a9d3-7e954b97ffa7 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:46.979126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 278/528] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:20:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:46.986492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250deb6d-531f-4668-bcd6-6e2f590dc0bc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:47.040356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d330a-77aa-461b-b469-bd9218097dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:47.041031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d330a-77aa-461b-b469-bd9218097dee None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:47.041311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d330a-77aa-461b-b469-bd9218097dee None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:47.041581 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d330a-77aa-461b-b469-bd9218097dee None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/a0a7230c6da74e6484db3f260b4c58df/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:47.047476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d330a-77aa-461b-b469-bd9218097dee None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=a0a7230c6da74e6484db3f260b4c58df, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:47.048934 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d330a-77aa-461b-b469-bd9218097dee None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:47.137446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 251/529] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:46 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/a0a7230c6da74e6484db3f260b4c58df/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:20:47.567431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d91db-c4c9-4793-a530-ef015baea247 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:47.568490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d91db-c4c9-4793-a530-ef015baea247 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:47.568980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d91db-c4c9-4793-a530-ef015baea247 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:47.571470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 279/530] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:47.576948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081656-e40e-4eb6-9466-9500f674c1e7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:47.577472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081656-e40e-4eb6-9466-9500f674c1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:47.577994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081656-e40e-4eb6-9466-9500f674c1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:47.600634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b7081656-e40e-4eb6-9466-9500f674c1e7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:47.602705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b7081656-e40e-4eb6-9466-9500f674c1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:47.691464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 252/531] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:47.819249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971b6f1-fd90-4620-a4e5-ff85376a6785 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:47.819547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971b6f1-fd90-4620-a4e5-ff85376a6785 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:47.819769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971b6f1-fd90-4620-a4e5-ff85376a6785 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:47.821893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 280/532] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:47.827278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad56e2-06f9-4281-8121-a9dc083f1525 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:47.827564 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad56e2-06f9-4281-8121-a9dc083f1525 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:47.827833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad56e2-06f9-4281-8121-a9dc083f1525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:47.846536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-efad56e2-06f9-4281-8121-a9dc083f1525 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:47.847879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-efad56e2-06f9-4281-8121-a9dc083f1525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:47.901823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:47.943104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 253/533] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:48.018362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971b6f1-fd90-4620-a4e5-ff85376a6785 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:48.136920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:48.148776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971b6f1-fd90-4620-a4e5-ff85376a6785 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:48.195582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86fb4c-cef8-4dd5-8aa8-925172814c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:48.196216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86fb4c-cef8-4dd5-8aa8-925172814c58 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:48.196498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86fb4c-cef8-4dd5-8aa8-925172814c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:48.196774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86fb4c-cef8-4dd5-8aa8-925172814c58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:48.207441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86fb4c-cef8-4dd5-8aa8-925172814c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-2d86fb4c-cef8-4dd5-8aa8-925172814c58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:48.212425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:48.213332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:48.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 281/534] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:48.219578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad56e2-06f9-4281-8121-a9dc083f1525 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:48.357708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad56e2-06f9-4281-8121-a9dc083f1525 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:48.364488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86fb4c-cef8-4dd5-8aa8-925172814c58 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:48.442770 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:48.443725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:48.444219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:48.444669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:48.446085 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:48.446599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:48.448148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:48.462184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 254/535] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:48.464912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:48.516402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:48.594816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86fb4c-cef8-4dd5-8aa8-925172814c58 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:48.614251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4494-ace2-41c7-842b-dd062fa5cb47 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:48.657475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:48.658153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:48.658433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:48.658705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:48.682558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f584d5-8e9f-47db-b922-6bed974d6b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:48.683201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f584d5-8e9f-47db-b922-6bed974d6b05 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:48.683476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f584d5-8e9f-47db-b922-6bed974d6b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:48.683743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f584d5-8e9f-47db-b922-6bed974d6b05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:48.688348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:48.689808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:48.705142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f584d5-8e9f-47db-b922-6bed974d6b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-f2f584d5-8e9f-47db-b922-6bed974d6b05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:48.710147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:48.711110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:48.711934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 282/536] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:48.717174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f584d5-8e9f-47db-b922-6bed974d6b05 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:48.718042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:48.720320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:48.722482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:48.722482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:48.722482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:48.722482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 255/537] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:48.732572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:48.748953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:48.864229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f584d5-8e9f-47db-b922-6bed974d6b05 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:48.864583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:48.884899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089c266-9fb5-43d2-8a50-320a76c0c1a6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:48.900959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bb947-3890-4103-8b84-0528955f722e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:48.901631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bb947-3890-4103-8b84-0528955f722e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:48.901921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bb947-3890-4103-8b84-0528955f722e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:48.902214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bb947-3890-4103-8b84-0528955f722e None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:48.902879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bb947-3890-4103-8b84-0528955f722e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:48.903179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bb947-3890-4103-8b84-0528955f722e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:48.904533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bb947-3890-4103-8b84-0528955f722e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:48.913583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 283/538] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:48.915393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01794-f018-4b7a-a2c0-bcab631d0cb1 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:48.915672 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01794-f018-4b7a-a2c0-bcab631d0cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:48.915939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01794-f018-4b7a-a2c0-bcab631d0cb1 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:48.917401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 284/539] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:48.923273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8709-aa15-48d7-9c79-2c5218747f19 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:48.923556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8709-aa15-48d7-9c79-2c5218747f19 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:48.923896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8709-aa15-48d7-9c79-2c5218747f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:48.931698 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:48.933233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:48.933519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:48.933794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:48.934493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:48.934797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:48.936157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:48.943712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-3eff8709-aa15-48d7-9c79-2c5218747f19 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:48.945258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3eff8709-aa15-48d7-9c79-2c5218747f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:48.950814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 256/540] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:48.951716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.020131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 285/541] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:49.021363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff8709-aa15-48d7-9c79-2c5218747f19 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.030939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:49.091362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e079dd8-fd9e-474e-acdd-e530a2b22fd4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:49.131177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff8709-aa15-48d7-9c79-2c5218747f19 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:49.142599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:49.145586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.146108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.146525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.165263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:49.167203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:49.180097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:49.194519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:49.202353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:49.203433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:49.203433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:49.203433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:49.203433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 257/542] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:49.203715 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:49.204680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.205356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.205795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.206615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.223932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:49.225607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:49.249786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:49.250992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:20:49.252103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:49.254058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 286/543] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Oct 12 12:20:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:49.262608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.378414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:49.395107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9085c3-4535-44c6-9528-0b3b31eb26bd None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:49.437921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55702515-e39c-4980-a451-77948447a403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:49.438836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55702515-e39c-4980-a451-77948447a403 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.439273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55702515-e39c-4980-a451-77948447a403 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.440175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55702515-e39c-4980-a451-77948447a403 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.441240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55702515-e39c-4980-a451-77948447a403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:49.441694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55702515-e39c-4980-a451-77948447a403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:49.443951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55702515-e39c-4980-a451-77948447a403 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:49.452479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64612872-89c1-482d-a11a-20324fa5e8aa None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:49.464423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 258/544] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:49.464877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55702515-e39c-4980-a451-77948447a403 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.496823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:49.497222 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.497479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.497755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.498556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:49.498867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:49.500365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:49.520469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 287/545] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Oct 12 12:20:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:49.521333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.546851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:49.600376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55702515-e39c-4980-a451-77948447a403 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:49.634601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212a37e-d7ea-4cf6-91c1-2813dd57cdec None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:49.640562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a64c2-d578-4e66-8346-c7963ad5f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:49.641221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a64c2-d578-4e66-8346-c7963ad5f5f0 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.641509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a64c2-d578-4e66-8346-c7963ad5f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.641778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a64c2-d578-4e66-8346-c7963ad5f5f0 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.642901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a64c2-d578-4e66-8346-c7963ad5f5f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:49.644643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a64c2-d578-4e66-8346-c7963ad5f5f0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:49.659176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:49.669558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:49.670213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.670548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.670789 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.677658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:49.679142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:49.690702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:49.691673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 288/546] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:49.692613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.724676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 259/547] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:20:49 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:49.726052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8985e9-1ce5-448e-acd0-9ffe7df36f51 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.726373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8985e9-1ce5-448e-acd0-9ffe7df36f51 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.726652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8985e9-1ce5-448e-acd0-9ffe7df36f51 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.729979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 260/548] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:49.729979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8985e9-1ce5-448e-acd0-9ffe7df36f51 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.817508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35487a32-e42c-4b72-a4fb-6105934066ba None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:49.862741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adcf4-923a-4c53-9496-ff5ba2cd5124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:49.863662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adcf4-923a-4c53-9496-ff5ba2cd5124 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.864094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adcf4-923a-4c53-9496-ff5ba2cd5124 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.864410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adcf4-923a-4c53-9496-ff5ba2cd5124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.870256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8985e9-1ce5-448e-acd0-9ffe7df36f51 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:49.876786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627adcf4-923a-4c53-9496-ff5ba2cd5124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:49.878293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627adcf4-923a-4c53-9496-ff5ba2cd5124 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:49.894712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-627adcf4-923a-4c53-9496-ff5ba2cd5124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:49.894887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:49.896087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:49.897002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:49.897002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:49.897002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:49.897002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 289/549] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:49.902848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d8c0-f12c-490e-a12c-e5f7fac509d0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.902848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d8c0-f12c-490e-a12c-e5f7fac509d0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.902848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314d8c0-f12c-490e-a12c-e5f7fac509d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.917268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec36796-2682-45b4-906c-56a366e3a34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:49.918402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec36796-2682-45b4-906c-56a366e3a34d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.918923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec36796-2682-45b4-906c-56a366e3a34d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.919153 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a314d8c0-f12c-490e-a12c-e5f7fac509d0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:49.919624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec36796-2682-45b4-906c-56a366e3a34d None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.920368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a314d8c0-f12c-490e-a12c-e5f7fac509d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:49.920789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec36796-2682-45b4-906c-56a366e3a34d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:49.921294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec36796-2682-45b4-906c-56a366e3a34d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:49.923306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec36796-2682-45b4-906c-56a366e3a34d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:49.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 261/550] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Oct 12 12:20:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:49.947138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b1c53-06cf-405c-b6b2-0fac5ddb3069 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:49.947427 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b1c53-06cf-405c-b6b2-0fac5ddb3069 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:49.947696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b1c53-06cf-405c-b6b2-0fac5ddb3069 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:49.948885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 262/551] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:49.949570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b1c53-06cf-405c-b6b2-0fac5ddb3069 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:49.951007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:50.000579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 290/552] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:50.001390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314d8c0-f12c-490e-a12c-e5f7fac509d0 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.065395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b1c53-06cf-405c-b6b2-0fac5ddb3069 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.102118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a180637-56e5-4fa4-836b-c202a7c11501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.102727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180637-56e5-4fa4-836b-c202a7c11501 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.103054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180637-56e5-4fa4-836b-c202a7c11501 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.103330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a180637-56e5-4fa4-836b-c202a7c11501 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.104028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a180637-56e5-4fa4-836b-c202a7c11501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:50.104334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a180637-56e5-4fa4-836b-c202a7c11501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.105705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a180637-56e5-4fa4-836b-c202a7c11501 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:50.109107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314d8c0-f12c-490e-a12c-e5f7fac509d0 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.122086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 263/553] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:50.123415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8425-70ef-4fce-90d9-75c76db237e6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.123701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8425-70ef-4fce-90d9-75c76db237e6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.123988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def8425-70ef-4fce-90d9-75c76db237e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.145515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8def8425-70ef-4fce-90d9-75c76db237e6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:50.145791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.146457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.146715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8def8425-70ef-4fce-90d9-75c76db237e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:50.146821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.146992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.157946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.159495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:50.176070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:50.177034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:50.177908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:50.177908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:50.177908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:50.178107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:50.178598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 291/554] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:50.179012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.196171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:50.225960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 264/555] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:20:50.225960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8425-70ef-4fce-90d9-75c76db237e6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.297239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227c534-9825-409a-ac1b-dbad4c567009 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.323912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def8425-70ef-4fce-90d9-75c76db237e6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.343407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.344244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.344685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.345515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.355014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.357292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:50.360642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.361412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.361788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.362214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.362944 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:50.363311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.364761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:50.371830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:50.372860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:50.373911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:50.373911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:50.373911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:50.373911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:50.373911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:50.373911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:50.373911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:50.373911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:50.374965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 292/556] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:50.376527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.381193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 265/557] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:50.385778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.410329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:50.475933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:50.500737 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b879a7-9ee4-4213-a894-911cfc45236d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.506014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f509a2-b372-4f17-9407-f1cbccc39909 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.542589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.543443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.543723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.543995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.544095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43543149-5604-49b6-a911-df494e7acad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.544651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543149-5604-49b6-a911-df494e7acad7 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.544750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:50.544928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543149-5604-49b6-a911-df494e7acad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.545025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.545198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543149-5604-49b6-a911-df494e7acad7 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.545840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43543149-5604-49b6-a911-df494e7acad7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.546400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:50.547207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43543149-5604-49b6-a911-df494e7acad7 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:50.559059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 266/558] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:50.559877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.623432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 293/559] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:20:50 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:50.624549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43543149-5604-49b6-a911-df494e7acad7 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.651564 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:50.671429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972aa28-6d22-49bd-8f52-14ef4b406764 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.727596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.728339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.728712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.729024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.738638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.740571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:50.754548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:50.755655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:50.756809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:50.756809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:50.756809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:50.756809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:50.756809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:50.756809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:50.756809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:50.756809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:50.757837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 267/560] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:50.761961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.765750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43543149-5604-49b6-a911-df494e7acad7 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.805001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752583b-970d-441f-b81f-b4919c6de5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.805698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752583b-970d-441f-b81f-b4919c6de5af None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.806000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752583b-970d-441f-b81f-b4919c6de5af None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.806272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752583b-970d-441f-b81f-b4919c6de5af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.808708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:50.819169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752583b-970d-441f-b81f-b4919c6de5af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-3752583b-970d-441f-b81f-b4919c6de5af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:50.824173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:50.826260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 294/561] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:50.828195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752583b-970d-441f-b81f-b4919c6de5af None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:50.886253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09113962-e2ec-49a7-bbc0-fca45b4af6b2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.927658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd19764-6629-4482-b6db-b3a6efac4441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.928337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd19764-6629-4482-b6db-b3a6efac4441 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.928618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd19764-6629-4482-b6db-b3a6efac4441 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.928890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd19764-6629-4482-b6db-b3a6efac4441 None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/3d937cb6b9054233b587fd1c39a9a662/roles/7fcd00bd7c614a9587d3161df27b1cff` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.944179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:50.948857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd19764-6629-4482-b6db-b3a6efac4441 None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=3d937cb6b9054233b587fd1c39a9a662, role_id=7fcd00bd7c614a9587d3161df27b1cff)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.954028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd19764-6629-4482-b6db-b3a6efac4441 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:50.954158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752583b-970d-441f-b81f-b4919c6de5af None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:50.989640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:50.990375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:50.990646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:50.990913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:50.991651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:50.991988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:50.993412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:51.006933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 295/562] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:51.012018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:51.029146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 268/563] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:50 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/3d937cb6b9054233b587fd1c39a9a662/roles/7fcd00bd7c614a9587d3161df27b1cff => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:51.030256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd19764-6629-4482-b6db-b3a6efac4441 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:51.137960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:51.200957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd19764-6629-4482-b6db-b3a6efac4441 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:51.242552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26eec56-1939-4928-b077-fbe1e646e085 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:51.244808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caacbf-432b-4378-a5d2-2585ce37d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:51.245484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caacbf-432b-4378-a5d2-2585ce37d8fc None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:51.245758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caacbf-432b-4378-a5d2-2585ce37d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:51.246054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caacbf-432b-4378-a5d2-2585ce37d8fc None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/36af83e8ee4048d78d2db8784f6347df/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:51.251944 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19caacbf-432b-4378-a5d2-2585ce37d8fc None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=36af83e8ee4048d78d2db8784f6347df, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:51.253400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19caacbf-432b-4378-a5d2-2585ce37d8fc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:51.290245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:51.290837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:51.291114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:51.291673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:51.292025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:51.292325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:51.294471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:51.303386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 296/564] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:51.308841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:51.354161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 269/565] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:51 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/36af83e8ee4048d78d2db8784f6347df/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:51.433177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52328282-9960-4860-a6d8-4c9f5a7f1174 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:51.451768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:51.472948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:51.473846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:51.474147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:51.474469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:51.491249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:51.493314 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:51.513109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:51.514121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:51.515004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:51.515004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:51.515004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:51.515004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 297/566] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:51.520330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caacbf-432b-4378-a5d2-2585ce37d8fc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:51.610395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:51.666786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caacbf-432b-4378-a5d2-2585ce37d8fc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:51.703322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:51.703843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:51.704118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:51.704445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:51.705093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:51.705447 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:51.706884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:51.721977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 270/567] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:51.727718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:51.835687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c0366-7e33-4ce5-bebb-8c1c8549587d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:51.877323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7ef42-c9e1-4ca5-90a9-06f327aea8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:51.877323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ef42-c9e1-4ca5-90a9-06f327aea8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:51.877323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ef42-c9e1-4ca5-90a9-06f327aea8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:51.877323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ef42-c9e1-4ca5-90a9-06f327aea8c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:51.891046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7ef42-c9e1-4ca5-90a9-06f327aea8c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:51.892833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7ef42-c9e1-4ca5-90a9-06f327aea8c2 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-8ea7ef42-c9e1-4ca5-90a9-06f327aea8c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:51.908977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:51.909990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:20:51.910977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:51.913534 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 298/568] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Oct 12 12:20:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:51.919462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:52.065837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6d6cd-ef09-4431-a9b1-5713f14a09be None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:52.071540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb491ba-a1f2-4656-b0fe-cd107fd89065 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:52.071828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb491ba-a1f2-4656-b0fe-cd107fd89065 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:52.072105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb491ba-a1f2-4656-b0fe-cd107fd89065 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:52.073582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 299/569] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:52.083290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3613-d328-4bec-9328-48473fde0867 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:52.083609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3613-d328-4bec-9328-48473fde0867 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:52.083885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3613-d328-4bec-9328-48473fde0867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:52.085414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:52.116660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a7ad3613-d328-4bec-9328-48473fde0867 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:52.118505 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a7ad3613-d328-4bec-9328-48473fde0867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:52.121191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:52.121706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:52.121994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:52.122263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:52.122995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:52.123296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:52.124758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:52.164315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 271/570] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Oct 12 12:20:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:52.165920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:52.168502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:52.258713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 300/571] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:52.314793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860e8a9-5e77-436a-8554-ad56f1ffbb44 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:52.349914 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d8e3e-ad21-44dc-b89d-23b3f4bed36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:52.350898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d8e3e-ad21-44dc-b89d-23b3f4bed36c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:52.351437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d8e3e-ad21-44dc-b89d-23b3f4bed36c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:52.351934 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d8e3e-ad21-44dc-b89d-23b3f4bed36c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:52.406345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:52.425906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d8e3e-ad21-44dc-b89d-23b3f4bed36c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:52.427819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d8e3e-ad21-44dc-b89d-23b3f4bed36c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:52.532991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 272/572] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:20:52.542069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad3613-d328-4bec-9328-48473fde0867 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:52.587896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d8e3e-ad21-44dc-b89d-23b3f4bed36c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:52.678571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:52.696574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d8e3e-ad21-44dc-b89d-23b3f4bed36c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:52.734742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:52.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:52.736232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:52.736703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:52.737481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:52.739102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:52.750088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:52.751103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:52.751963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:52.756033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 273/573] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Oct 12 12:20:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:52.767305 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:52.815763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad3613-d328-4bec-9328-48473fde0867 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:52.879862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:52.880485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:52.880759 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:52.881032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:52.900818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:52.902294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:52.922854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:52.923846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:52.924711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:52.924711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:52.924711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:52.932714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 301/574] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:52.935259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:52.955535 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f40dcb-2643-4f4d-a2a7-79a8ca2d08f0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:52.961871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:53.018272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:53.018877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.019155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.019427 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.020055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:53.020356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:53.021731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:53.026250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:53.031081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 274/575] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Oct 12 12:20:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:53.036662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:53.113451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cb337-e2aa-431e-9ea5-4841379210e0 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:53.149778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c370f0-e10c-40bd-ba3b-1c07cce81815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:53.150514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c370f0-e10c-40bd-ba3b-1c07cce81815 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.150796 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c370f0-e10c-40bd-ba3b-1c07cce81815 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.151067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c370f0-e10c-40bd-ba3b-1c07cce81815 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.151708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c370f0-e10c-40bd-ba3b-1c07cce81815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:53.152114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c370f0-e10c-40bd-ba3b-1c07cce81815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:53.153747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c370f0-e10c-40bd-ba3b-1c07cce81815 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:53.159090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84fcbe-e171-4de5-b7b1-8a214e907061 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:53.166811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 302/576] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:53.168123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04eb0f-305a-492a-b63c-6bb90634f1c9 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.168414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04eb0f-305a-492a-b63c-6bb90634f1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.168737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04eb0f-305a-492a-b63c-6bb90634f1c9 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.174138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 303/577] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:53.175765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02332-adcc-4846-8643-33f6125bdadf None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.176115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02332-adcc-4846-8643-33f6125bdadf None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.176456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02332-adcc-4846-8643-33f6125bdadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.195649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-53f02332-adcc-4846-8643-33f6125bdadf None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:53.196978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-53f02332-adcc-4846-8643-33f6125bdadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:53.199303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:53.199977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.200259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.200528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.201160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:53.201462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:53.202842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:53.214191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 275/578] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Oct 12 12:20:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:53.215044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:53.273967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:53.274530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 304/579] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:20:53.275828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f02332-adcc-4846-8643-33f6125bdadf None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:53.278043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:53.354167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385533b-a49e-4b09-8b87-0003b01d86cb None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:53.391688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:53.391688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.391688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.391688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.396922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:53.399086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:53.409405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f02332-adcc-4846-8643-33f6125bdadf None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:53.410492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:53.411369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:53.412479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:53.412479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:53.412479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:53.412479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:53.412479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:53.412479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:53.412479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:53.412479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:53.416329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 276/580] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:53.417975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:53.484680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:53.485368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.485653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.486054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.486833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] RBAC: Adding query filter params (interface=public, service_id=119bffe100934b6c9c32231efd840e4d, region_id=RegionOne) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:53.487149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:53.488519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:53.502904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 305/581] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Oct 12 12:20:53 2022] GET /identity/v3/endpoints?service_id=119bffe100934b6c9c32231efd840e4d&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:53.518434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:53.527127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005451c6-53b2-4879-a2ac-d0bf0ca189cf None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:53.550249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:53.561691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:53.562343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.562625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.562898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.563586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:53.563912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:53.565361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:53.584080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 277/582] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:53.585319 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:53.658458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:53.689266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95380669-4ce2-49fb-ae94-8cc6fa574da3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:53.725121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:53.726162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.726556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.726906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.727858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=a0a7230c6da74e6484db3f260b4c58df) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:53.728241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:53.729753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:53.742746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 278/583] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:53 2022] GET /identity/v3/role_assignments?user.id=a0a7230c6da74e6484db3f260b4c58df&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:53.744753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd30a-e6b1-4072-a177-f1548ae85bca None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:53.792825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d49d2-c140-4d31-8f8f-d54032749bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:53.793451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d49d2-c140-4d31-8f8f-d54032749bcf None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.793729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d49d2-c140-4d31-8f8f-d54032749bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.794030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d49d2-c140-4d31-8f8f-d54032749bcf None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.794646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d49d2-c140-4d31-8f8f-d54032749bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:53.796021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d49d2-c140-4d31-8f8f-d54032749bcf None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:53.801986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 306/584] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:20:53 2022] GET /identity/v3/services => generated 772 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:53.863510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:53.885080 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b594b-eb08-4f4e-844e-d6954fe660db None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.885651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b594b-eb08-4f4e-844e-d6954fe660db None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.885651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b594b-eb08-4f4e-844e-d6954fe660db None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.887087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 307/585] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:53.893249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f936-bb32-42c2-9500-c4aa22c95bc9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:53.893593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f936-bb32-42c2-9500-c4aa22c95bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:53.893793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f936-bb32-42c2-9500-c4aa22c95bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:53.930797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2425f936-bb32-42c2-9500-c4aa22c95bc9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:53.932128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2425f936-bb32-42c2-9500-c4aa22c95bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:53.946431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:54.016230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3477f-ec52-4b1f-8989-783409100d2c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:54.045303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 308/586] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:54.058364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d61d1-6cee-4ca7-a478-5f795caef221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:54.059425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d61d1-6cee-4ca7-a478-5f795caef221 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.059910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d61d1-6cee-4ca7-a478-5f795caef221 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.060383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d61d1-6cee-4ca7-a478-5f795caef221 None admin] PATH_INFO: `/v3/services/network` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.061213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d61d1-6cee-4ca7-a478-5f795caef221 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:54.062872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d61d1-6cee-4ca7-a478-5f795caef221 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-545d61d1-6cee-4ca7-a478-5f795caef221 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:54.080986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:54.082078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:54.083171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 279/587] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:53 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:54.090262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425f936-bb32-42c2-9500-c4aa22c95bc9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:54.158531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:54.160491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05a49-3eb4-4fc7-9b5f-93df98bedfb5 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.160781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05a49-3eb4-4fc7-9b5f-93df98bedfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.161062 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05a49-3eb4-4fc7-9b5f-93df98bedfb5 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.162673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 280/588] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:54.168464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003af37-592d-4790-8db4-5cd3b18ebf03 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.168752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003af37-592d-4790-8db4-5cd3b18ebf03 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.170078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003af37-592d-4790-8db4-5cd3b18ebf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.196457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e003af37-592d-4790-8db4-5cd3b18ebf03 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:54.197684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e003af37-592d-4790-8db4-5cd3b18ebf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:54.205364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425f936-bb32-42c2-9500-c4aa22c95bc9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:54.252509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:54.253123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.254140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.254416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.255282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] RBAC: Adding query filter params (name=network) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:54.255587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:54.258269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:54.269316 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 309/589] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:54.280427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:54.336543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 281/590] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:20:54.345422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003af37-592d-4790-8db4-5cd3b18ebf03 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:54.407811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb71e09-b972-4bfe-b5aa-80e1c27fea57 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:54.411264 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:54.435602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:54.448026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:54.448640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.448921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.449191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.449818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] RBAC: Adding query filter params (type=network) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:54.450171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:54.455102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:54.468401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 310/591] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:54.474786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:54.515477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003af37-592d-4790-8db4-5cd3b18ebf03 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:54.586182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80497c9-4dbf-46f3-8101-477313faaaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:54.586815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80497c9-4dbf-46f3-8101-477313faaaea None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.587073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80497c9-4dbf-46f3-8101-477313faaaea None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.587347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80497c9-4dbf-46f3-8101-477313faaaea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.612375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80497c9-4dbf-46f3-8101-477313faaaea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-c80497c9-4dbf-46f3-8101-477313faaaea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:54.618608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:54.619517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:20:54.620327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:54.625840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 282/592] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:54.627802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80497c9-4dbf-46f3-8101-477313faaaea None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:54.637612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9d30-bbb9-485b-9a0f-7f1e793aff68 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:54.691446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:54.692533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.692858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.693135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.693843 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] RBAC: Adding query filter params (interface=public, service_id=f4c10713dbcf4aff95b54795bc286b22, region_id=RegionOne) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:54.694176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:54.696869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:54.704333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 311/593] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/endpoints?service_id=f4c10713dbcf4aff95b54795bc286b22&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:54.705283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:54.752536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:54.756279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80497c9-4dbf-46f3-8101-477313faaaea None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:54.772558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:54.791442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:54.792328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.792776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.793191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.794126 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:54.794571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:54.796072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:54.806500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 283/594] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:54.807582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:54.835950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fcd7b-0ad7-499c-a66d-a3fcc284bf77 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:54.881162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ccc09-9e24-4bca-809f-ff27459b8125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:54.881720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ccc09-9e24-4bca-809f-ff27459b8125 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.882023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ccc09-9e24-4bca-809f-ff27459b8125 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.882301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ccc09-9e24-4bca-809f-ff27459b8125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.895981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ccc09-9e24-4bca-809f-ff27459b8125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-683ccc09-9e24-4bca-809f-ff27459b8125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:54.903336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:54.904269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:54.905123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 312/595] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:54.905815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ccc09-9e24-4bca-809f-ff27459b8125 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:54.930217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af472699-e032-4f88-ab76-53b744efb183 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:54.978721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2159aa6-567e-4dd1-8059-7a3a12b085a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:54.979329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2159aa6-567e-4dd1-8059-7a3a12b085a1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:54.979609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2159aa6-567e-4dd1-8059-7a3a12b085a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:54.979887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2159aa6-567e-4dd1-8059-7a3a12b085a1 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:54.980654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2159aa6-567e-4dd1-8059-7a3a12b085a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:54.982609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2159aa6-567e-4dd1-8059-7a3a12b085a1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:54.989057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 284/596] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:54.989980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2159aa6-567e-4dd1-8059-7a3a12b085a1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:55.026764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ccc09-9e24-4bca-809f-ff27459b8125 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:55.027388 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:55.052398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:55.110404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:55.111068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:55.111342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:55.111610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:55.139208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:55.140657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:55.172071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:55.173073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:55.174010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:55.174010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:55.174010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:55.176698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 313/597] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:55.179497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:55.189202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2159aa6-567e-4dd1-8059-7a3a12b085a1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:55.236215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:55.237186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:55.237671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:55.238218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:55.239130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:55.239507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:55.240948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:55.256164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 285/598] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:55.266095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:55.328117 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a5ce2-8a6a-4797-a527-f8007d268537 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:55.399078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:55.399973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:55.400430 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:55.400864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36397-5cbf-4fed-bf4f-99a53f185d8b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:55.401066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:55.401878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:55.402412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:55.409959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:55.410101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:55.428488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 314/599] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:55.434313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:55.459162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:55.459778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:55.460054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:55.460325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:55.480626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:55.481679 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:55.483928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:55.512239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:55.513242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:55.514171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:55.514171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:55.514171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:55.514612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 286/600] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:55.520584 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:55.631112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21149509-0d1a-487d-8ced-9bce2341db9f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:55.667432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdb69b-2751-4bbe-9b4a-310412139d6f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:55.695466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:55.696200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:55.696487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:55.696768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:55.703769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:55.704364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:55.704639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:55.704907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:55.705545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:55.705843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:55.707244 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:55.711078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:55.712516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:55.719861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:55.724471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 287/601] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:55.725331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:55.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:55.745330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:55.747639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 315/602] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Oct 12 12:20:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:55.750442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:55.830019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6aec1-585b-4062-bbb9-063008d5dd3d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:55.866586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:55.868283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:55.868643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:55.868912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:55.876900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:55.878394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:20:55.891751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:55.892726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:55.893643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 288/603] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:55.898725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:55.962412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:55.994309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:56.024033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6ff3-7b60-40d6-b102-b646a00eab10 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:56.029313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b791ea-2480-44f3-842e-a45fac843e12 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:56.092070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:56.092678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.092954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.093221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.093974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:56.094284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:56.095713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:56.101029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:56.102355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.102636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.102905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.103631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:56.103941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:56.105341 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:56.138252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 316/604] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Oct 12 12:20:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:56.140226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 289/605] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Oct 12 12:20:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:56.145906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:56.147142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:56.257968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11f94b-f251-4e3e-b421-0e2ed2887359 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:56.278374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:56.298408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e359f-d368-40b8-a892-f15e3dd8b3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:56.299243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e359f-d368-40b8-a892-f15e3dd8b3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.299741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e359f-d368-40b8-a892-f15e3dd8b3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.300109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e359f-d368-40b8-a892-f15e3dd8b3b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.342745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a0e71-2964-4144-8630-39eecdb839ea None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:56.377608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e359f-d368-40b8-a892-f15e3dd8b3b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:56.379385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e359f-d368-40b8-a892-f15e3dd8b3b9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:56.388922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d26297-a423-4f38-bbb9-fc03c546e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:56.389738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d26297-a423-4f38-bbb9-fc03c546e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.390210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d26297-a423-4f38-bbb9-fc03c546e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.390711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d26297-a423-4f38-bbb9-fc03c546e4e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.403920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:56.466010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d26297-a423-4f38-bbb9-fc03c546e4e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:56.467575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d26297-a423-4f38-bbb9-fc03c546e4e5 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:56.490785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 290/606] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:20:56.490785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f81a3c-a49c-402d-836b-36e6cd406081 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.490785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f81a3c-a49c-402d-836b-36e6cd406081 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.490785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f81a3c-a49c-402d-836b-36e6cd406081 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.490785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 291/607] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:56.490785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb8b63-3cf3-403a-8149-f50dfeb87f20 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.491306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb8b63-3cf3-403a-8149-f50dfeb87f20 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.491306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb8b63-3cf3-403a-8149-f50dfeb87f20 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.492652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 292/608] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:56.497221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb8b63-3cf3-403a-8149-f50dfeb87f20 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:56.517102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:56.589771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 317/609] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:20:56.589771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e8aa8-f164-4b34-b125-2184aed90955 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.589771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e8aa8-f164-4b34-b125-2184aed90955 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.589771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e8aa8-f164-4b34-b125-2184aed90955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.605141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-733e8aa8-f164-4b34-b125-2184aed90955 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:56.606360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-733e8aa8-f164-4b34-b125-2184aed90955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:56.610558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb8b63-3cf3-403a-8149-f50dfeb87f20 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:56.640348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:56.647561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:56.648747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.649035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.652473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.659592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:56.661020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:56.681967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 318/610] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:56.682137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:56.683531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:56.684598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:56.684598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:56.684598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:56.684807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bc058-2ba3-4f1c-8b56-e092559d5842 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.684902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 293/611] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:56.684984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bc058-2ba3-4f1c-8b56-e092559d5842 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.684984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bc058-2ba3-4f1c-8b56-e092559d5842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.685128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:56.712615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-041bc058-2ba3-4f1c-8b56-e092559d5842 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:56.713966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-041bc058-2ba3-4f1c-8b56-e092559d5842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:56.769327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:56.798124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 319/612] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:56.799034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bc058-2ba3-4f1c-8b56-e092559d5842 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:56.820672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8e23c-3f71-45e6-8b87-3283edabd315 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:56.881840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:56.882556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:56.883325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:56.883598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:56.901694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:56.904286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:20:56.932479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:56.933459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:20:56.934370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:20:56.934370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:20:56.934370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:56.934370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 294/613] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:56.939938 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:56.962391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:56.966128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bc058-2ba3-4f1c-8b56-e092559d5842 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.012183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.013720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.014223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.014691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.015530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:57.016027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.017684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.034068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 320/614] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:57.035162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.080656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc63e68-78fb-46ef-9523-8dcde21664d4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.105136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:57.118886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.119495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.119778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.120052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.120693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:57.120998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.122431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.135337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 295/615] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:57.136189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.165654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4f302-fb74-428b-975d-af97a2c8a808 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.202607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.203219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.203491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.203754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.204325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.205671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:57.214888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:57.215825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:57.218972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 321/616] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:57.219767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.243353 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:57.248682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dc132-49ab-4467-a633-f18512dca0d9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.287181 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.287819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.288059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.288332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.296571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.298700 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:57.311749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:57.312650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 296/617] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:57.313548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.326567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59cbde-97f2-43bb-88c5-2643ee584892 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.362200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:57.366960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.367607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.367939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.368215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.368815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.370230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:57.379869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:57.386306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:57.387497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 322/618] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:57.388281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.431600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816e78a-3a0d-4c0e-8535-522249ffc352 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.467835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.468458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.468729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.469004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.475852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.477308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.477779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:20:57.488223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:57.489291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:57.490224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:20:57.490224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:20:57.490224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:20:57.490224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:20:57.490224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:20:57.490224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:20:57.490224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:20:57.490224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:20:57.490670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b24bb-4aaa-4ce7-b9dd-6cd412ca37ee None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.490743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 297/619] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:57.490743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.523870 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.524530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.524810 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.525106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.525730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:57.526054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.527409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.531798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 323/620] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:57.532603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.593024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae01cea-e69b-4dd1-a2a1-b38836fc1428 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.594494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:57.626300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18a5dd-db98-4544-8330-be98ea33c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.626959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18a5dd-db98-4544-8330-be98ea33c3ff None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.627240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18a5dd-db98-4544-8330-be98ea33c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.627518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18a5dd-db98-4544-8330-be98ea33c3ff None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.628206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18a5dd-db98-4544-8330-be98ea33c3ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:57.628512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18a5dd-db98-4544-8330-be98ea33c3ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.629563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fe41-7c92-4148-8234-a6bf92469a72 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.629966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18a5dd-db98-4544-8330-be98ea33c3ff None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.642185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 298/621] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:57.643432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f49b8-57d4-4549-b6dc-105cf1cb5cff None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.643718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f49b8-57d4-4549-b6dc-105cf1cb5cff None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.643987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f49b8-57d4-4549-b6dc-105cf1cb5cff None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.645534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 299/622] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:57.646527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f49b8-57d4-4549-b6dc-105cf1cb5cff None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.668266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.668858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.669134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.669403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.670049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:57.670352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.671696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.676103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 324/623] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:57.676722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.694555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:57.750158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f49b8-57d4-4549-b6dc-105cf1cb5cff None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.775169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2d2e2-cb82-43a1-a57f-6169c005867b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.783297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.783890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.784169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.784438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.785119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:57.785481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.786927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.799043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 300/624] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:57.799818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.812356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c6837-59e9-4345-b3d7-6593b9e63a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.812954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c6837-59e9-4345-b3d7-6593b9e63a5d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.813229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c6837-59e9-4345-b3d7-6593b9e63a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.813508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c6837-59e9-4345-b3d7-6593b9e63a5d None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.814149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c6837-59e9-4345-b3d7-6593b9e63a5d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:57.814455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c6837-59e9-4345-b3d7-6593b9e63a5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.815783 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c6837-59e9-4345-b3d7-6593b9e63a5d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.833963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 325/625] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:57.833963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888b3c0-694f-4283-8029-2b764a7283fc None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.833963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888b3c0-694f-4283-8029-2b764a7283fc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.833963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888b3c0-694f-4283-8029-2b764a7283fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.843093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:57.857218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b888b3c0-694f-4283-8029-2b764a7283fc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:57.858683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b888b3c0-694f-4283-8029-2b764a7283fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:57.908551 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95d620-9b75-44e4-852e-c1c40fa170a1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:57.937027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 326/626] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:57.937904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888b3c0-694f-4283-8029-2b764a7283fc None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:57.951246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:57.966003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:57.966659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:57.966938 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:57.967210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:57.967971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=36af83e8ee4048d78d2db8784f6347df) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:57.968297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:57.969726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:57.982467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 301/627] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/role_assignments?user.id=36af83e8ee4048d78d2db8784f6347df&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:57.982467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:58.141453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888b3c0-694f-4283-8029-2b764a7283fc None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:58.175251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:58.190558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe91dd-1d8d-4f76-a512-9a60f221c08a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:58.199953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:58.200656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:58.200942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:58.201214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:58.201854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:58.202203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:58.203980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:58.210573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 327/628] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:58.212264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:58.245640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:58.246282 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:58.246559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:58.246822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:58.247547 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] RBAC: Adding query filter params (role.id=7fcd00bd7c614a9587d3161df27b1cff, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=3d937cb6b9054233b587fd1c39a9a662) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:58.247850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:58.249265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:58.255697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 302/629] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:57 2022] GET /identity/v3/role_assignments?user.id=3d937cb6b9054233b587fd1c39a9a662&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7fcd00bd7c614a9587d3161df27b1cff => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:58.256321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:58.328524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:58.365911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea909a-cb3b-49ce-b36f-aecebd0a09f3 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:58.371271 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d49b17-3b40-4730-bb7b-5b3c0468e894 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:58.412347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311be9f5-be66-4b56-8432-54b1490b2eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:58.412915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311be9f5-be66-4b56-8432-54b1490b2eb1 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:58.413227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311be9f5-be66-4b56-8432-54b1490b2eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:58.413458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311be9f5-be66-4b56-8432-54b1490b2eb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:58.414234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311be9f5-be66-4b56-8432-54b1490b2eb1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:58.415713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311be9f5-be66-4b56-8432-54b1490b2eb1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:58.417432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:58.418424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:58.418935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:58.419560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] PATH_INFO: `/v3/services/network` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:58.420348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:58.422785 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:58.434579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:58.435686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:20:58.436556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:20:58.439557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 328/630] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:20:58 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:20:58.446099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:58.532309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 303/631] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:20:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:58.586388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1914394-901d-4bd4-9215-5688d6c41e32 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:58.634820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:58.635822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:58.636307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:58.636792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:58.637636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] RBAC: Adding query filter params (name=network) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:58.638330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:58.639931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:58.641416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:58.647093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 329/632] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:20:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:20:58.653527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311be9f5-be66-4b56-8432-54b1490b2eb1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:58.664986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:58.774862 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311be9f5-be66-4b56-8432-54b1490b2eb1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:58.814160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4470a0-6965-43b7-89a6-3bd1a8ccd6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:58.815221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4470a0-6965-43b7-89a6-3bd1a8ccd6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:58.815742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4470a0-6965-43b7-89a6-3bd1a8ccd6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:58.816216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4470a0-6965-43b7-89a6-3bd1a8ccd6a1 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:58.817045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4470a0-6965-43b7-89a6-3bd1a8ccd6a1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:58.817547 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4470a0-6965-43b7-89a6-3bd1a8ccd6a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:58.819203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4470a0-6965-43b7-89a6-3bd1a8ccd6a1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:58.825099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 304/633] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:20:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:58.830925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:58.970493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec212343-13de-4d65-bb89-3b327801915e None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:59.020513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf7bc2-1abc-422e-b39b-0b729494d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:59.021119 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf7bc2-1abc-422e-b39b-0b729494d502 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:59.021396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf7bc2-1abc-422e-b39b-0b729494d502 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:59.021667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf7bc2-1abc-422e-b39b-0b729494d502 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:59.022372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf7bc2-1abc-422e-b39b-0b729494d502 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:59.024237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf7bc2-1abc-422e-b39b-0b729494d502 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:59.137893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 330/634] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:20:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:00.257100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613772-db3b-4137-89af-7967e139c840 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.258196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613772-db3b-4137-89af-7967e139c840 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.258196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b613772-db3b-4137-89af-7967e139c840 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:00.259670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 305/635] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:00.267286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35437-2326-41fd-b648-8df6e953b2e7 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.268087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35437-2326-41fd-b648-8df6e953b2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.268564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35437-2326-41fd-b648-8df6e953b2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:00.287668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-89d35437-2326-41fd-b648-8df6e953b2e7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:00.289196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-89d35437-2326-41fd-b648-8df6e953b2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:00.345217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a280b7-b4e1-4a6b-9512-fac75ed01459 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.345217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a280b7-b4e1-4a6b-9512-fac75ed01459 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.345217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a280b7-b4e1-4a6b-9512-fac75ed01459 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:00.345217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 306/636] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:00.345590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:00.345675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b3864-c9f5-469d-a469-307667c1efef None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.345675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b3864-c9f5-469d-a469-307667c1efef None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.345675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b3864-c9f5-469d-a469-307667c1efef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:00.361527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e31b3864-c9f5-469d-a469-307667c1efef None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:00.362857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e31b3864-c9f5-469d-a469-307667c1efef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:00.369308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 331/637] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:00.433067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:00.438287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 307/638] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:00.638120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d35437-2326-41fd-b648-8df6e953b2e7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:00.709502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b3864-c9f5-469d-a469-307667c1efef None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:00.741382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d35437-2326-41fd-b648-8df6e953b2e7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:00.781678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:00.782356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.782641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.782916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:00.783509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:00.785018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:00.793886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:00.794953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:00.795884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 332/639] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:00.803461 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:00.839736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b3864-c9f5-469d-a469-307667c1efef None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:00.862364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:00.878106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74322866-4fcf-4d9a-abca-2716968d11ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:00.880671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74322866-4fcf-4d9a-abca-2716968d11ad None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.880671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74322866-4fcf-4d9a-abca-2716968d11ad None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.880671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74322866-4fcf-4d9a-abca-2716968d11ad None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:00.880671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74322866-4fcf-4d9a-abca-2716968d11ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:00.881576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74322866-4fcf-4d9a-abca-2716968d11ad None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:00.915979 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbcee4-4770-4fd3-8e8f-d91593a7dd9b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:00.948969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883830e-1953-4a9f-aaa6-82a353712a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:00.949568 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883830e-1953-4a9f-aaa6-82a353712a90 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.949842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883830e-1953-4a9f-aaa6-82a353712a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.950195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883830e-1953-4a9f-aaa6-82a353712a90 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:00.955236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883830e-1953-4a9f-aaa6-82a353712a90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:00.955236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883830e-1953-4a9f-aaa6-82a353712a90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:00.955236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883830e-1953-4a9f-aaa6-82a353712a90 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:00.956994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 333/640] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:00.958179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae52473-97da-42a1-a10c-711dc4f7e3f3 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.958458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae52473-97da-42a1-a10c-711dc4f7e3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.965422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae52473-97da-42a1-a10c-711dc4f7e3f3 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:00.965422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 334/641] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:00.965422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae52473-97da-42a1-a10c-711dc4f7e3f3 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:00.965999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 308/642] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:00 2022] POST /identity/v3/services => generated 269 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:21:00.967615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9121f-4b5e-4626-8574-e0efa9bdac44 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:00.968010 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9121f-4b5e-4626-8574-e0efa9bdac44 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:00.968383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9121f-4b5e-4626-8574-e0efa9bdac44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.000125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1cf9121f-4b5e-4626-8574-e0efa9bdac44 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:01.002990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1cf9121f-4b5e-4626-8574-e0efa9bdac44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:01.088340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:01.105772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae52473-97da-42a1-a10c-711dc4f7e3f3 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:01.109765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:01.115906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 309/643] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:00 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:01.137168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0774c-ad02-424e-849d-b99ce82dc977 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:01.137511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0774c-ad02-424e-849d-b99ce82dc977 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:01.137704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0774c-ad02-424e-849d-b99ce82dc977 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.139064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 310/644] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:01.144558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794387a-2ed5-4530-9828-54696fca000e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:01.144894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794387a-2ed5-4530-9828-54696fca000e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:01.145168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794387a-2ed5-4530-9828-54696fca000e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.154722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:01.155328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:01.155608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:01.155931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.156576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:01.157115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:01.159014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:01.164775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 335/645] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:01.170687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d794387a-2ed5-4530-9828-54696fca000e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:01.170687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d794387a-2ed5-4530-9828-54696fca000e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:01.257226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 311/646] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:01.392318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:01.496799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e5f25-a536-4b40-b155-5f7cd3517b68 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:01.529357 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794387a-2ed5-4530-9828-54696fca000e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:01.532772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104efac-daea-469f-8456-0cd5f9fff3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:01.533408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104efac-daea-469f-8456-0cd5f9fff3dd None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:01.533687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104efac-daea-469f-8456-0cd5f9fff3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:01.533989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104efac-daea-469f-8456-0cd5f9fff3dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.544939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104efac-daea-469f-8456-0cd5f9fff3dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-9104efac-daea-469f-8456-0cd5f9fff3dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:01.549728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:01.550815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:21:01.551698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:01.556294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 336/647] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Oct 12 12:21:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:01.557546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104efac-daea-469f-8456-0cd5f9fff3dd None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:01.588198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:01.644821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794387a-2ed5-4530-9828-54696fca000e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:01.661689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:01.671165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104efac-daea-469f-8456-0cd5f9fff3dd None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:01.681045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:01.681775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:01.682090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:01.682366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.682952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:01.684431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:01.693755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:01.694787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:01.696844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 312/648] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:01.702501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:01.714145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:01.714838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:01.715121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:01.715387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.716152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:01.716454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:01.717794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:01.727577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 337/649] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Oct 12 12:21:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:01.733064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:01.808032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acc8d4-351c-408a-980f-806b6bf8458a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:01.844282 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:01.844888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:01.845165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:01.845437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.846094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:01.846399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:01.846662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:01.847759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:01.853037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 313/650] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:01.858758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:01.859664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a405f-fb22-45e8-86d0-7b8b98639019 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:01.905626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:01.907176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:01.907693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:01.908621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:01.915598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:01.920526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:01.921985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:01.937411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:01.938445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:01.939365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:01.939365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:01.939365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:01.942194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 338/651] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:01.944441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:01.986099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9120c4-f3d1-4617-b96f-49f83bb831bd None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:02.029283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:02.030781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:02.031548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:02.032053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:02.033040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:02.033644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:02.035400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:02.043376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 314/652] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:02.059478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69570248-c54e-4238-a22a-f77e7043b4c9 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:02.098720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:02.099337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:02.099616 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:02.099945 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:02.100599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:02.100960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:02.102418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:02.117317 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 339/653] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:02.123091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:02.234207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2eb77f-c3fc-4536-832c-3c30f206c65d None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:02.243885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:02.269118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:02.269741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:02.269741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:02.269876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:02.279193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:02.280634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:02.291987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:02.292941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:02.293927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 315/654] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Oct 12 12:21:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:02.303231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:02.306621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:02.410622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16263434-78ff-4d9c-94ea-78f1a261f7e7 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:02.447060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:02.447722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:02.448003 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:02.448320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:02.449053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:02.449360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:02.450801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:02.469246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 340/655] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Oct 12 12:21:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:02.475269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:02.576788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a2a3d-eaa8-4bf6-86dc-695faed54d15 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:02.611046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:02.611669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:02.611949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:02.612221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:02.623478 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:02.624910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:02.639189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:02.640115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:02.640985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:02.640985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:02.640985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:02.640985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 316/656] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:02.650109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:02.712383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:02.756541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946f63-7b8e-4c8a-a9f0-7c943ca5162e None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:02.791657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:02.792262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:02.792540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:02.792813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:02.793447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:02.793748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:02.795138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:02.811867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 341/657] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:02.813241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:02.837297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:02.920031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e7001-24af-483c-964d-08f69c08e309 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:02.954331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:02.955201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:02.955647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:02.956072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:02.963754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:02.965357 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:02.977618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:02.978703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:02.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:02.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:02.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:02.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:02.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:02.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:02.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:02.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:02.981767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 317/658] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:02.987433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:03.099221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430a821-5d93-4d19-b0bf-aac5ff21e6dc None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:03.120634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:03.135171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392e539-a81e-4035-9921-5c493dbf3542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:03.135780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e539-a81e-4035-9921-5c493dbf3542 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:03.136058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e539-a81e-4035-9921-5c493dbf3542 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:03.136331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392e539-a81e-4035-9921-5c493dbf3542 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:03.137022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392e539-a81e-4035-9921-5c493dbf3542 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:03.137327 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392e539-a81e-4035-9921-5c493dbf3542 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:03.138771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392e539-a81e-4035-9921-5c493dbf3542 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:03.151356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 342/659] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:03.157261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:03.247545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:03.267733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1b236-2446-49d4-a364-09d66d3b47a8 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:03.307342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926283e1-f62e-43e0-bc9b-5c5950b40eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:03.308160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926283e1-f62e-43e0-bc9b-5c5950b40eb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:03.308563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926283e1-f62e-43e0-bc9b-5c5950b40eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:03.308841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926283e1-f62e-43e0-bc9b-5c5950b40eb2 None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/3d937cb6b9054233b587fd1c39a9a662/roles/390d6cb1aefc46ea875435aa9fded217` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:03.310284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c968f-ea72-43d6-aeb7-b493b405d3a2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:03.310768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c968f-ea72-43d6-aeb7-b493b405d3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:03.311217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926283e1-f62e-43e0-bc9b-5c5950b40eb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=3d937cb6b9054233b587fd1c39a9a662, role_id=390d6cb1aefc46ea875435aa9fded217)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:03.311326 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c968f-ea72-43d6-aeb7-b493b405d3a2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:03.312729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926283e1-f62e-43e0-bc9b-5c5950b40eb2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:03.312967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 343/660] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:03.318370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed133d8e-706a-474c-a7f6-0ca2844a8d5c None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:03.318658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed133d8e-706a-474c-a7f6-0ca2844a8d5c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:03.318932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed133d8e-706a-474c-a7f6-0ca2844a8d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:03.338106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ed133d8e-706a-474c-a7f6-0ca2844a8d5c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:03.339275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ed133d8e-706a-474c-a7f6-0ca2844a8d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:03.383973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 318/661] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:21:03 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/3d937cb6b9054233b587fd1c39a9a662/roles/390d6cb1aefc46ea875435aa9fded217 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:21:03.405004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85a5b-62c1-419c-9186-bf16a784cf7c None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:03.405229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85a5b-62c1-419c-9186-bf16a784cf7c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:03.405406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85a5b-62c1-419c-9186-bf16a784cf7c None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:03.407222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 319/662] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:03.416483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cf52f-b809-4eca-8f76-1debbe9007dc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:03.416781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cf52f-b809-4eca-8f76-1debbe9007dc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:03.417186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cf52f-b809-4eca-8f76-1debbe9007dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:03.423152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 344/663] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:03.444196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4b6cf52f-b809-4eca-8f76-1debbe9007dc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:03.445947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4b6cf52f-b809-4eca-8f76-1debbe9007dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:03.527261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 320/664] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:03.699105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed133d8e-706a-474c-a7f6-0ca2844a8d5c None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:03.807420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed133d8e-706a-474c-a7f6-0ca2844a8d5c None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:03.818397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cf52f-b809-4eca-8f76-1debbe9007dc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:03.818808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:03.843529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dda9e1-f68e-4659-b5b2-28794e286ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:03.844134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dda9e1-f68e-4659-b5b2-28794e286ea2 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:03.844416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dda9e1-f68e-4659-b5b2-28794e286ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:03.844692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dda9e1-f68e-4659-b5b2-28794e286ea2 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:03.845379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dda9e1-f68e-4659-b5b2-28794e286ea2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:03.846756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dda9e1-f68e-4659-b5b2-28794e286ea2 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:03.900154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:03.930362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cf52f-b809-4eca-8f76-1debbe9007dc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:03.957685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 345/665] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:03 2022] POST /identity/v3/services => generated 241 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:03.980147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:03.980809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:03.981090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:03.981361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:03.982007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:03.984169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:03.994860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:03.995926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:03.997222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 321/666] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Oct 12 12:21:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:04.002110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dda9e1-f68e-4659-b5b2-28794e286ea2 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:04.124433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dda9e1-f68e-4659-b5b2-28794e286ea2 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:04.162285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:04.162898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:04.163180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:04.163447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:04.164123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:04.164433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:04.165796 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:04.170884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 346/667] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:04.176696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:04.284163 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48718d19-f1f8-4d84-9530-6c01b677dfb0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:04.320586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:04.325353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7eb26-9061-4d7e-8b7e-8397a0a80467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:04.326047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7eb26-9061-4d7e-8b7e-8397a0a80467 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:04.326338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7eb26-9061-4d7e-8b7e-8397a0a80467 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:04.326615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7eb26-9061-4d7e-8b7e-8397a0a80467 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:04.327277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7eb26-9061-4d7e-8b7e-8397a0a80467 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:04.327587 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7eb26-9061-4d7e-8b7e-8397a0a80467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:04.329065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7eb26-9061-4d7e-8b7e-8397a0a80467 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:04.335375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 322/668] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:04.341814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:04.379594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:04.431532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8660-59fc-4753-a688-a2353959d344 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:04.432511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8660-59fc-4753-a688-a2353959d344 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:04.432812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8660-59fc-4753-a688-a2353959d344 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:04.434172 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 323/669] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:04.443297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc478f8-1c1d-4612-ab75-cb0de413ca7d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:04.443571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc478f8-1c1d-4612-ab75-cb0de413ca7d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:04.443840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc478f8-1c1d-4612-ab75-cb0de413ca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:04.458769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750ec5e-6ad7-4f26-b272-4d5045913e38 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:04.464005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-abc478f8-1c1d-4612-ab75-cb0de413ca7d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:04.465195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-abc478f8-1c1d-4612-ab75-cb0de413ca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:04.497452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:04.498463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:04.498996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:04.499460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:04.500462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] RBAC: Adding query filter params (interface=public, service_id=76c8485588804564865b206fee8f81f8, region_id=RegionOne) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:04.500967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:04.502559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:04.508393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 347/670] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Oct 12 12:21:04 2022] GET /identity/v3/endpoints?service_id=76c8485588804564865b206fee8f81f8&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:04.515348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:04.552257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 324/671] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:04 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:04.623482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:04.627883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3779d-9b23-4845-915e-739f6ffdc603 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:04.664410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87732e72-d6ca-4f78-a01a-fccc5af8afe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:04.665014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732e72-d6ca-4f78-a01a-fccc5af8afe4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:04.665370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732e72-d6ca-4f78-a01a-fccc5af8afe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:04.665643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87732e72-d6ca-4f78-a01a-fccc5af8afe4 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:04.666275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87732e72-d6ca-4f78-a01a-fccc5af8afe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:04.667628 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87732e72-d6ca-4f78-a01a-fccc5af8afe4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:04.673808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 348/672] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:21:04 2022] GET /identity/v3/services => generated 1258 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:21:04.834912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc478f8-1c1d-4612-ab75-cb0de413ca7d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:04.948162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc478f8-1c1d-4612-ab75-cb0de413ca7d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:04.973631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:04.984720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ad3bf-ab21-4125-9681-b675ff58098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:04.985504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ad3bf-ab21-4125-9681-b675ff58098d None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:04.985926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ad3bf-ab21-4125-9681-b675ff58098d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:04.986356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ad3bf-ab21-4125-9681-b675ff58098d None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:04.987159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ad3bf-ab21-4125-9681-b675ff58098d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:04.991465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ad3bf-ab21-4125-9681-b675ff58098d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:05.090736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 325/673] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:04 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:05.620618 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae712-b229-4448-b713-eff17d9c5860 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:05.621160 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae712-b229-4448-b713-eff17d9c5860 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:05.621160 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadae712-b229-4448-b713-eff17d9c5860 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:05.622763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 349/674] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:05.628079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aba318-4c9a-42a7-89d9-df6632f55076 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:05.628515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aba318-4c9a-42a7-89d9-df6632f55076 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:05.628691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aba318-4c9a-42a7-89d9-df6632f55076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:05.647666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-47aba318-4c9a-42a7-89d9-df6632f55076 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:05.648885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-47aba318-4c9a-42a7-89d9-df6632f55076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:05.724361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 326/675] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:05.993612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadae712-b229-4448-b713-eff17d9c5860 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:06.111459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadae712-b229-4448-b713-eff17d9c5860 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:06.150626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082a61b-fb27-43ac-83b6-242e5606dbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:06.151525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082a61b-fb27-43ac-83b6-242e5606dbee None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.151951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082a61b-fb27-43ac-83b6-242e5606dbee None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.152383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082a61b-fb27-43ac-83b6-242e5606dbee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.163085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082a61b-fb27-43ac-83b6-242e5606dbee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-4082a61b-fb27-43ac-83b6-242e5606dbee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:06.168082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:06.169214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:21:06.170323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:06.172582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 350/676] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Oct 12 12:21:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:06.179029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aba318-4c9a-42a7-89d9-df6632f55076 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:06.221901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac89cd-fdd6-417b-9ad1-bdd23fcc68fc None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.222477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac89cd-fdd6-417b-9ad1-bdd23fcc68fc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.222755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac89cd-fdd6-417b-9ad1-bdd23fcc68fc None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.226226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 351/677] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:06.230382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef15b-964b-48f2-ab76-990c2e74d019 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.230661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef15b-964b-48f2-ab76-990c2e74d019 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.230931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef15b-964b-48f2-ab76-990c2e74d019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.251553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-34bef15b-964b-48f2-ab76-990c2e74d019 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:06.252674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-34bef15b-964b-48f2-ab76-990c2e74d019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:06.275582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:06.293976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aba318-4c9a-42a7-89d9-df6632f55076 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:06.307689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:06.337195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 352/678] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:06.340618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:06.341229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.341506 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.341777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.342471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:06.342774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:06.344207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:06.353489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 327/679] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Oct 12 12:21:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:06.359390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef15b-964b-48f2-ab76-990c2e74d019 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:06.466912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef15b-964b-48f2-ab76-990c2e74d019 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:06.507094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:06.508190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.508470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.508740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.519382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:06.520818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:06.535891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:06.536885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:06.538404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:06.538404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:06.538404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:06.538404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 353/680] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:06.542971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:06.643200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:06.646314 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a7dc-687b-4bc1-95cb-567abd4ca4ee None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:06.674290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:06.682485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:06.683332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.683612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.683883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.684520 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:06.684867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:06.686244 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:06.699333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 328/681] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:06.705229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:06.742576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:06.769837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6678c-20cd-43a8-9a52-acd167404956 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:06.821321 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:06.822390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.823103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.823813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.824550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:06.826227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:06.833968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7d8e2-fa87-44a4-aa74-57746fca6bd9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:06.837986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:06.839095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:06.839940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:06.842696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 354/682] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:06.848375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:06.881877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80fa73-ac4d-452b-9581-362377987773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:06.882721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80fa73-ac4d-452b-9581-362377987773 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.883279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80fa73-ac4d-452b-9581-362377987773 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.883697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80fa73-ac4d-452b-9581-362377987773 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.892671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80fa73-ac4d-452b-9581-362377987773 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:06.894888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80fa73-ac4d-452b-9581-362377987773 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1c80fa73-ac4d-452b-9581-362377987773 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:06.910303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:06.911285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:21:06.912231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:06.913466 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 329/683] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Oct 12 12:21:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:06.915421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166893e-de37-46ee-a500-52626eaf4fde None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:06.915767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166893e-de37-46ee-a500-52626eaf4fde None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:06.916139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166893e-de37-46ee-a500-52626eaf4fde None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:06.917969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 330/684] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:06.918712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166893e-de37-46ee-a500-52626eaf4fde None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:06.936816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:06.972177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec9a1c-eace-401b-a5b4-9ef047a9d0f8 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.008481 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe7e8f-0b65-45e6-99ef-adbc75d5dea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.009093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe7e8f-0b65-45e6-99ef-adbc75d5dea6 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.009372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe7e8f-0b65-45e6-99ef-adbc75d5dea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.009648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe7e8f-0b65-45e6-99ef-adbc75d5dea6 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.010307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe7e8f-0b65-45e6-99ef-adbc75d5dea6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:07.010613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe7e8f-0b65-45e6-99ef-adbc75d5dea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.011972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe7e8f-0b65-45e6-99ef-adbc75d5dea6 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.016948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 355/685] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:07.018206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400bac-8a92-4f9f-99a0-b43d5a21be3e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.018510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400bac-8a92-4f9f-99a0-b43d5a21be3e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.018765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400bac-8a92-4f9f-99a0-b43d5a21be3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.030257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:07.037656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d8400bac-8a92-4f9f-99a0-b43d5a21be3e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:07.039373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d8400bac-8a92-4f9f-99a0-b43d5a21be3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:07.041025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166893e-de37-46ee-a500-52626eaf4fde None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.082537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.083149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.083431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.083703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.084440 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:07.084745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.086235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.104031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 331/686] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Oct 12 12:21:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:07.104851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.130239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 356/687] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:07.132120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8400bac-8a92-4f9f-99a0-b43d5a21be3e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.206254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:07.230511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbebc9-e021-4326-b57d-2d4912fcf69a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.250682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8400bac-8a92-4f9f-99a0-b43d5a21be3e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.287936 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.289455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.289865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058bffe-58ad-4acb-af14-4fc5551f740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.290104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.290589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.290785 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058bffe-58ad-4acb-af14-4fc5551f740a None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.291152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058bffe-58ad-4acb-af14-4fc5551f740a None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.291511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058bffe-58ad-4acb-af14-4fc5551f740a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.291837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:07.292401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.293913 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.300775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 332/688] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:07.306241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.327829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:07.370364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058bffe-58ad-4acb-af14-4fc5551f740a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.371794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058bffe-58ad-4acb-af14-4fc5551f740a None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.423332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07decd-0122-4bc7-8ce4-2a0bbe0d9281 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.435593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:07.457302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9200112-e539-4f17-a06b-1499c8e2f162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.457914 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200112-e539-4f17-a06b-1499c8e2f162 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.458385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200112-e539-4f17-a06b-1499c8e2f162 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.458666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200112-e539-4f17-a06b-1499c8e2f162 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.459422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9200112-e539-4f17-a06b-1499c8e2f162 None admin] RBAC: Adding query filter params (interface=public, service_id=9288211263cb4ae2be62f6533fcce7b1, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:07.459733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9200112-e539-4f17-a06b-1499c8e2f162 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.461622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9200112-e539-4f17-a06b-1499c8e2f162 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.468832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 333/689] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/endpoints?service_id=9288211263cb4ae2be62f6533fcce7b1&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:07.470168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc810b-bc3b-4a34-a89a-c2ca00e3c805 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.470456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc810b-bc3b-4a34-a89a-c2ca00e3c805 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.470731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc810b-bc3b-4a34-a89a-c2ca00e3c805 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.471935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 334/690] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:07.474212 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc810b-bc3b-4a34-a89a-c2ca00e3c805 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.482071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 357/691] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:21:07.482071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058bffe-58ad-4acb-af14-4fc5551f740a None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.553661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:07.575269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc810b-bc3b-4a34-a89a-c2ca00e3c805 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.578397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058bffe-58ad-4acb-af14-4fc5551f740a None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.609218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.609842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.610155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.610432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.611013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.612369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.612732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45b27e-4362-42ba-ad53-dd711219d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.613332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45b27e-4362-42ba-ad53-dd711219d7ce None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.613612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45b27e-4362-42ba-ad53-dd711219d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.613884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45b27e-4362-42ba-ad53-dd711219d7ce None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.614515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45b27e-4362-42ba-ad53-dd711219d7ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.615858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45b27e-4362-42ba-ad53-dd711219d7ce None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:07.621301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:07.622434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:21:07.623342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:07.624064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 358/692] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/services => generated 1497 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:21:07.624134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 335/693] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:07.624134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.625383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a0f4e-8d83-4cbf-939e-5357923c6d0b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.625839 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a0f4e-8d83-4cbf-939e-5357923c6d0b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.626296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a0f4e-8d83-4cbf-939e-5357923c6d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.656793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-132a0f4e-8d83-4cbf-939e-5357923c6d0b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:07.658136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-132a0f4e-8d83-4cbf-939e-5357923c6d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:07.680471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:07.733252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 359/694] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:07.734275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a0f4e-8d83-4cbf-939e-5357923c6d0b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.746397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0ac0f-8d7f-4d0b-993b-0bd13068abfe None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.785794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.786749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.787192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.787627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.798638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.800936 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.803138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:07.823550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:07.824500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:07.825751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:07.825751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:07.825751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:07.825751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 336/695] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:07.833998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.870270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a0f4e-8d83-4cbf-939e-5357923c6d0b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:07.909633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:07.910379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:07.910585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:07.910862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:07.911492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:07.911798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:07.913395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:07.919097 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 360/696] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:07.925127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:07.953849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:07.974287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1516de-5bc4-4214-89c8-388843bf6706 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.010996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.011598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.011879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.012153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.012796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:08.013091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.014527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.027499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 337/697] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:08.028746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.055073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ca1a3-a646-4ac8-9f10-2f2909422a7b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.094439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:08.096472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.097031 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.097310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.097577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.098238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:08.099067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.100217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.109852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 361/698] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:08.109852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.150490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692bf3a-42a8-4533-9ea6-fdfa410f4b79 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.187039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.187669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.187950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.188225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.188802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.190185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.200621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:08.201715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:08.202735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:08.203568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 338/699] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:08.204368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.221923 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fad38-4588-471c-95bf-ed49e7ac575d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.259764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.260377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.260647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.260919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.267580 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.268999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:08.280878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:08.281840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 362/700] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:08.283334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.318650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4751e4-f5f8-4a53-8a27-9d46699ccab4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.338693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:08.357772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0db8f-d17c-47b8-85dd-0dad345c7888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.358340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0db8f-d17c-47b8-85dd-0dad345c7888 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.358648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0db8f-d17c-47b8-85dd-0dad345c7888 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.359073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0db8f-d17c-47b8-85dd-0dad345c7888 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.363668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0db8f-d17c-47b8-85dd-0dad345c7888 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.363668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0db8f-d17c-47b8-85dd-0dad345c7888 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.387020 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6918e-61cc-45da-a6d0-dc0613f24804 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.419815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.420413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.420686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.420951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.421632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:08.421955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.423297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.427848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 363/701] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:08.427848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.443199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:08.474563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 339/702] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:21:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:08.475890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0db8f-d17c-47b8-85dd-0dad345c7888 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.543379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6a303-e2a0-40d1-91e4-87e20953d64f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.588579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.589192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.590175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.590477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.591196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:08.591509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.592965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.608128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 364/703] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:21:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:08.614665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.623675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0db8f-d17c-47b8-85dd-0dad345c7888 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.666761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.667644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.668140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.668563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.669364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:08.669909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.671841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.678177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 340/704] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:08.684783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.697254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:08.737348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:08.742633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1e4d2-d21e-4ab3-a443-40579d59ab3b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.785876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.786544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.786828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.787421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.788437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] RBAC: Adding query filter params (role.id=390d6cb1aefc46ea875435aa9fded217, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=3d937cb6b9054233b587fd1c39a9a662) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:08.789296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.791378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.799241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 365/705] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:21:08 2022] GET /identity/v3/role_assignments?user.id=3d937cb6b9054233b587fd1c39a9a662&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=390d6cb1aefc46ea875435aa9fded217 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:08.813846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2571b0-2bed-4fb3-bd9c-91e750082e84 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:08.853919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b4df-882b-43d8-a725-b07723dc18f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:08.854588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b4df-882b-43d8-a725-b07723dc18f9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:08.854872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b4df-882b-43d8-a725-b07723dc18f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:08.855146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b4df-882b-43d8-a725-b07723dc18f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:08.855865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0b4df-882b-43d8-a725-b07723dc18f9 None admin] RBAC: Adding query filter params (interface=public, service_id=79af401169c94f99bae7ecbc0a4ec78d, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:08.856176 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0b4df-882b-43d8-a725-b07723dc18f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:08.857618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0b4df-882b-43d8-a725-b07723dc18f9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:08.868821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 341/706] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Oct 12 12:21:08 2022] GET /identity/v3/endpoints?service_id=79af401169c94f99bae7ecbc0a4ec78d&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:08.870264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:08.988837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c46b5-1e94-4053-abd8-500ad6170ea0 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:09.017553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:09.026454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002e526-3817-4fd0-9502-a0b89df859f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:09.027072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e526-3817-4fd0-9502-a0b89df859f9 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:09.027353 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e526-3817-4fd0-9502-a0b89df859f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:09.027627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e526-3817-4fd0-9502-a0b89df859f9 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:09.028237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002e526-3817-4fd0-9502-a0b89df859f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:09.029600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002e526-3817-4fd0-9502-a0b89df859f9 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:09.036260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 366/707] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:21:08 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:21:09.826443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d6aa-4ba6-4cdf-ba9c-c1b40127b834 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:09.826970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d6aa-4ba6-4cdf-ba9c-c1b40127b834 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:09.826970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d6aa-4ba6-4cdf-ba9c-c1b40127b834 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:09.828545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 342/708] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:09.834020 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a779f4-1a1c-46bb-9b78-7fc66a5d77a2 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:09.834445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a779f4-1a1c-46bb-9b78-7fc66a5d77a2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:09.834590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a779f4-1a1c-46bb-9b78-7fc66a5d77a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:09.853983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b6a779f4-1a1c-46bb-9b78-7fc66a5d77a2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:09.855171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b6a779f4-1a1c-46bb-9b78-7fc66a5d77a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:09.925268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 367/709] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:10.196139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833d6aa-4ba6-4cdf-ba9c-c1b40127b834 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:10.304109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833d6aa-4ba6-4cdf-ba9c-c1b40127b834 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:10.305663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:10.340291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:10.341113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:10.341680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:10.341982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:10.342570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:10.343994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:10.354147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:10.355135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:10.355987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 343/710] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:10.361582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a779f4-1a1c-46bb-9b78-7fc66a5d77a2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:10.462130 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a779f4-1a1c-46bb-9b78-7fc66a5d77a2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:10.485690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:10.497283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:10.498594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:10.498594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:10.498594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:10.499070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:10.499376 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:10.500740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:10.505886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 368/711] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:10.511720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:10.615430 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae3505-6dd3-4ff3-8808-12778e9ae3d7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:10.652776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6554f97-586f-4222-b450-f33d4a0b0fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:10.653416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6554f97-586f-4222-b450-f33d4a0b0fbf None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:10.653698 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6554f97-586f-4222-b450-f33d4a0b0fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:10.654005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6554f97-586f-4222-b450-f33d4a0b0fbf None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:10.654670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6554f97-586f-4222-b450-f33d4a0b0fbf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:10.654974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6554f97-586f-4222-b450-f33d4a0b0fbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:10.656398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6554f97-586f-4222-b450-f33d4a0b0fbf None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:10.662685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 344/712] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:10.668480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:10.764332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ef40-4781-4ecc-96f7-e403fda37e39 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:10.764631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ef40-4781-4ecc-96f7-e403fda37e39 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:10.764818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7ef40-4781-4ecc-96f7-e403fda37e39 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:10.766480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 345/713] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:10.771585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d409f-c1c2-46f6-bee4-c1b15010fa6f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:10.771930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d409f-c1c2-46f6-bee4-c1b15010fa6f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:10.772212 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d409f-c1c2-46f6-bee4-c1b15010fa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:10.788008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5296326-5d86-4db7-95f6-d4251fdd3b7c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:10.792010 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-624d409f-c1c2-46f6-bee4-c1b15010fa6f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:10.793214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-624d409f-c1c2-46f6-bee4-c1b15010fa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:10.818992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:10.827515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96865978-8177-42eb-b974-021d25e9ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:10.828401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865978-8177-42eb-b974-021d25e9ccf7 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:10.828884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865978-8177-42eb-b974-021d25e9ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:10.829371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96865978-8177-42eb-b974-021d25e9ccf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:10.830297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96865978-8177-42eb-b974-021d25e9ccf7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:10.832894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96865978-8177-42eb-b974-021d25e9ccf7 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:10.869988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 346/714] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:10.910464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 369/715] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:21:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:11.046211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c62a-0e37-44b6-b8f9-dd0d00b32753 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.046865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c62a-0e37-44b6-b8f9-dd0d00b32753 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.047347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c62a-0e37-44b6-b8f9-dd0d00b32753 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.049291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 347/716] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:11.056189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068f442-7d6e-49ec-9db5-fe48410fafdd None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.056487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068f442-7d6e-49ec-9db5-fe48410fafdd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.056756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068f442-7d6e-49ec-9db5-fe48410fafdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.081627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-6068f442-7d6e-49ec-9db5-fe48410fafdd None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:11.083219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6068f442-7d6e-49ec-9db5-fe48410fafdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:11.121662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:11.157611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c62a-0e37-44b6-b8f9-dd0d00b32753 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:11.162156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 370/717] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:11.267129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c62a-0e37-44b6-b8f9-dd0d00b32753 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:11.303345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba399-ad02-4b13-940c-a32580c68a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:11.304434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba399-ad02-4b13-940c-a32580c68a13 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.304922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba399-ad02-4b13-940c-a32580c68a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.305422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba399-ad02-4b13-940c-a32580c68a13 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.306329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fba399-ad02-4b13-940c-a32580c68a13 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:11.307893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fba399-ad02-4b13-940c-a32580c68a13 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-86fba399-ad02-4b13-940c-a32580c68a13 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:11.317220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:11.318243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:11.319065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:11.321803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 348/718] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:11.328010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068f442-7d6e-49ec-9db5-fe48410fafdd None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:11.358841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0ffc7-b2f1-415d-8347-6c5e64c1560f None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.359146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0ffc7-b2f1-415d-8347-6c5e64c1560f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.359412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0ffc7-b2f1-415d-8347-6c5e64c1560f None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.360977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 349/719] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:11.365864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802860c7-4f2a-42d3-b514-fce98819eb3e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.366180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802860c7-4f2a-42d3-b514-fce98819eb3e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.366515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802860c7-4f2a-42d3-b514-fce98819eb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.385382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-802860c7-4f2a-42d3-b514-fce98819eb3e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:11.386677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-802860c7-4f2a-42d3-b514-fce98819eb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:11.405671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:11.438055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068f442-7d6e-49ec-9db5-fe48410fafdd None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:11.458854 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 350/720] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:11.459714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802860c7-4f2a-42d3-b514-fce98819eb3e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:11.477316 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:11.478413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.478951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.479419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.480234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:11.480730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:11.482434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:11.492971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 371/721] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:21:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:11.494098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:11.533987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:11.573039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802860c7-4f2a-42d3-b514-fce98819eb3e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:11.614726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:11.615502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.615876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.616242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.617198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:11.619389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:11.624826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27ee5d-3f09-45d9-9da5-16a9f10b8931 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:11.630144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:11.631352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:11.632899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:11.634884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 351/722] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Oct 12 12:21:11 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:11.640438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:11.662029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:11.677100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:11.677714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.678021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.678299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.678931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] RBAC: Adding query filter params (type=compute) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:11.679262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:11.680633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:11.685875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 372/723] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:21:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:11.762756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:11.766214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c6d1a-1bd7-47d9-905f-7b990b9f33d4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:11.808842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:11.809803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.810282 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.810703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.811506 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] RBAC: Adding query filter params (name=image) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:11.812067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:11.813726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:11.821636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 352/724] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:11.827932 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:11.891676 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:11.904860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34e1d1-d3c5-4519-8ae8-05daeb02cbea None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:11.934593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:11.952552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:11.953219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:11.953682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:11.954066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:11.954888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:11.956942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:11.958796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1caa7-0e8e-4b58-984f-d0fada12cdfe None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:11.967217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:11.968192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:21:11.969027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:11.971935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 373/725] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:11.978491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:12.008003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:12.008909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:12.009372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:12.010124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:12.011372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] RBAC: Adding query filter params (type=image) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:12.012174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:12.014036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:12.025958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 353/726] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:12.097943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0a1fb-a132-45b0-a72e-862f4334fcd4 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:12.137654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:12.138300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:12.138581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:12.138854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:12.139498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:12.139806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:12.141187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:12.147086 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 374/727] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:12.155170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:12.265981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df7f6c-48e8-4b37-97ce-0563743c6613 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:12.302069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f3873-ed06-4ad8-bbee-45707d6a5ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:12.303024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f3873-ed06-4ad8-bbee-45707d6a5ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:12.303506 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f3873-ed06-4ad8-bbee-45707d6a5ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:12.304000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f3873-ed06-4ad8-bbee-45707d6a5ff8 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:12.305378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f3873-ed06-4ad8-bbee-45707d6a5ff8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:12.305893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f3873-ed06-4ad8-bbee-45707d6a5ff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:12.307538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f3873-ed06-4ad8-bbee-45707d6a5ff8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:12.316917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 354/728] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Oct 12 12:21:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:12.319281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:12.351688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:12.428142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275f0fd-a927-46c4-a9a5-ffc081e85b97 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:12.464083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df866496-9bd4-4e78-a802-183819eca9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:12.464738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866496-9bd4-4e78-a802-183819eca9e7 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:12.465012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866496-9bd4-4e78-a802-183819eca9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:12.465279 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866496-9bd4-4e78-a802-183819eca9e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:12.466028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df866496-9bd4-4e78-a802-183819eca9e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:12.467429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df866496-9bd4-4e78-a802-183819eca9e7 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:12.550197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 375/729] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:21:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:13.186562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0566-48ca-4adf-8436-ec75eec8f84d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:13.187074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0566-48ca-4adf-8436-ec75eec8f84d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:13.187150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0566-48ca-4adf-8436-ec75eec8f84d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:13.188666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 355/730] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:13.194056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f74f0-d28a-4f3a-a8a9-ca7de425bceb None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:13.194456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f74f0-d28a-4f3a-a8a9-ca7de425bceb None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:13.194622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f74f0-d28a-4f3a-a8a9-ca7de425bceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:13.213738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-8b7f74f0-d28a-4f3a-a8a9-ca7de425bceb None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:13.214947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8b7f74f0-d28a-4f3a-a8a9-ca7de425bceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:13.288806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 376/731] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:13.562654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a0566-48ca-4adf-8436-ec75eec8f84d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:13.576685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:13.672866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a0566-48ca-4adf-8436-ec75eec8f84d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:13.708875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:13.709495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:13.709777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:13.710067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:13.710673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:13.712025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:13.720969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:13.724723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:13.728361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 356/732] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:13.731238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f74f0-d28a-4f3a-a8a9-ca7de425bceb None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:13.758334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:13.908304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f74f0-d28a-4f3a-a8a9-ca7de425bceb None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:13.973064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139394ca-2cb7-460f-ae7a-3165731a92b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:13.974393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139394ca-2cb7-460f-ae7a-3165731a92b9 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:13.974880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139394ca-2cb7-460f-ae7a-3165731a92b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:13.975352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139394ca-2cb7-460f-ae7a-3165731a92b9 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:13.976981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139394ca-2cb7-460f-ae7a-3165731a92b9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:13.977517 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139394ca-2cb7-460f-ae7a-3165731a92b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:13.980072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139394ca-2cb7-460f-ae7a-3165731a92b9 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:13.988978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 377/733] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:13.995199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:14.040914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c823-f868-4115-9ce0-b76e916eb938 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.041258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c823-f868-4115-9ce0-b76e916eb938 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.041447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9c823-f868-4115-9ce0-b76e916eb938 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.043135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 378/734] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:14.048391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f293b0-00d4-479c-a51b-5ef8601996a7 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.048682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f293b0-00d4-479c-a51b-5ef8601996a7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.049100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f293b0-00d4-479c-a51b-5ef8601996a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.080002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e0f293b0-00d4-479c-a51b-5ef8601996a7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:14.081401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e0f293b0-00d4-479c-a51b-5ef8601996a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:14.094785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:14.125746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e836ed9-c586-4ce1-bf22-b59f03e77d0c None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:14.167784 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:14.171482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:14.172167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.172584 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.172915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.173701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:14.174165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:14.176858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:14.178071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 379/735] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:14.186624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 357/736] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:14.195655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f293b0-00d4-479c-a51b-5ef8601996a7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:14.295711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f293b0-00d4-479c-a51b-5ef8601996a7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:14.331288 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:14.331890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.332167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.332437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.333138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] RBAC: Adding query filter params (interface=admin, service_id=9288211263cb4ae2be62f6533fcce7b1, region_id=RegionOne) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:14.333446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:14.334880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:14.344905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 380/737] 10.222.0.44 () {56 vars in 1249 bytes} [Wed Oct 12 12:21:14 2022] GET /identity/v3/endpoints?service_id=9288211263cb4ae2be62f6533fcce7b1&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:14.351130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:14.461096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56762440-3139-4252-ac7a-5798fd729dd2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:14.490269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:14.497504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa511efd-36b6-45a1-9439-a53502c5d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:14.498151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511efd-36b6-45a1-9439-a53502c5d9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.498420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511efd-36b6-45a1-9439-a53502c5d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.498691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa511efd-36b6-45a1-9439-a53502c5d9e7 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.499297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa511efd-36b6-45a1-9439-a53502c5d9e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:14.500666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa511efd-36b6-45a1-9439-a53502c5d9e7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:14.507810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 358/738] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:21:14 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:21:14.545327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3960-319e-4eb9-9e8b-9a632f83ae2b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.545664 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3960-319e-4eb9-9e8b-9a632f83ae2b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.545880 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3960-319e-4eb9-9e8b-9a632f83ae2b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.547665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 359/739] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:14.553508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b6e00-4e25-4079-ac1b-443fdea2d640 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.553839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b6e00-4e25-4079-ac1b-443fdea2d640 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.554087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b6e00-4e25-4079-ac1b-443fdea2d640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.561351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:14.576225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-df1b6e00-4e25-4079-ac1b-443fdea2d640 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:14.578100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-df1b6e00-4e25-4079-ac1b-443fdea2d640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:14.663391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 360/740] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:14.692794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:14.735521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17118b6-cf4e-4070-af56-b73d00fde340 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:14.771648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5130-282a-40f4-8987-93c6cc67af08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:14.772251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5130-282a-40f4-8987-93c6cc67af08 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.772525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5130-282a-40f4-8987-93c6cc67af08 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.772790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5130-282a-40f4-8987-93c6cc67af08 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.774047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e5130-282a-40f4-8987-93c6cc67af08 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:14.775435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e5130-282a-40f4-8987-93c6cc67af08 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:14.792831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7ea16-bb0c-4472-86d3-d28780020c38 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.793120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7ea16-bb0c-4472-86d3-d28780020c38 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.793386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7ea16-bb0c-4472-86d3-d28780020c38 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.794852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 361/741] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:14.799831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e09fb-b1c1-4a31-8d1d-8903305aa23b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:14.800108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e09fb-b1c1-4a31-8d1d-8903305aa23b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:14.800437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e09fb-b1c1-4a31-8d1d-8903305aa23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:14.818438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2f1e09fb-b1c1-4a31-8d1d-8903305aa23b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:14.819614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2f1e09fb-b1c1-4a31-8d1d-8903305aa23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:14.861186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 381/742] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:14 2022] POST /identity/v3/services => generated 242 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:14.908225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 362/743] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:14.939217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5130-282a-40f4-8987-93c6cc67af08 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:15.056890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5130-282a-40f4-8987-93c6cc67af08 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:15.087496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:15.100942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c0674-3a5c-4c4c-a7f8-71b52806d274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:15.102201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0674-3a5c-4c4c-a7f8-71b52806d274 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:15.102481 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0674-3a5c-4c4c-a7f8-71b52806d274 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:15.102747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c0674-3a5c-4c4c-a7f8-71b52806d274 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:15.103393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c0674-3a5c-4c4c-a7f8-71b52806d274 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:15.104741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c0674-3a5c-4c4c-a7f8-71b52806d274 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:15.171509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 382/744] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:14 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:15.192912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e09fb-b1c1-4a31-8d1d-8903305aa23b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:15.266038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:15.314711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e09fb-b1c1-4a31-8d1d-8903305aa23b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:15.360700 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:15.361362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:15.361641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:15.361992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:15.362641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:15.364170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:15.366832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING keystone.server.flask.common [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 12 12:21:15.420340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.sql.core [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 12:21:15.420340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:21:15.420340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [parameters: {'id': 'd4ffa03df4d54756ba495cfc33c5f1a1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 12 12:21:15.420340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4ffa03df4d54756ba495cfc33c5f1a1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4ffa03df4d54756ba495cfc33c5f1a1. Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:21:15.428251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 12:21:15.429038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application yield resource Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise self.value Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application session.commit() Oct 12 12:21:15.430030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:21:15.430929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:21:15.431817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application result.read() Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:21:15.432614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [parameters: {'id': 'd4ffa03df4d54756ba495cfc33c5f1a1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:15.433462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:15.434258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d4ffa03df4d54756ba495cfc33c5f1a1. Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:15.435174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 363/745] 10.222.0.44 () {60 vars in 1140 bytes} [Wed Oct 12 12:21:15 2022] POST /identity/v3/projects => generated 262 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:15.437161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c0674-3a5c-4c4c-a7f8-71b52806d274 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:15.550648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c0674-3a5c-4c4c-a7f8-71b52806d274 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:15.587565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48dc9d-d7df-421f-9ca4-ff99faeb5a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:15.588108 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48dc9d-d7df-421f-9ca4-ff99faeb5a92 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:15.588379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48dc9d-d7df-421f-9ca4-ff99faeb5a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:15.588644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48dc9d-d7df-421f-9ca4-ff99faeb5a92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:15.595942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48dc9d-d7df-421f-9ca4-ff99faeb5a92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:15.597363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48dc9d-d7df-421f-9ca4-ff99faeb5a92 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-8b48dc9d-d7df-421f-9ca4-ff99faeb5a92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:15.609115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:15.611129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:15.613302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 383/746] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:15.617589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:15.721767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d7780-92e0-4fe0-93d4-c55784888890 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:15.733261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:15.765842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5cc91b-5171-4fed-a1d5-7aa7abab4c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:15.766493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cc91b-5171-4fed-a1d5-7aa7abab4c9d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:15.766778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cc91b-5171-4fed-a1d5-7aa7abab4c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:15.767055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5cc91b-5171-4fed-a1d5-7aa7abab4c9d None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:15.767720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5cc91b-5171-4fed-a1d5-7aa7abab4c9d None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:15.768028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5cc91b-5171-4fed-a1d5-7aa7abab4c9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:15.769660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5cc91b-5171-4fed-a1d5-7aa7abab4c9d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:15.790493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 364/747] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:21:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:16.754187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0c5d2-742c-44bb-b1fb-9e1ba5b1f041 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:16.754713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0c5d2-742c-44bb-b1fb-9e1ba5b1f041 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:16.754713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0c5d2-742c-44bb-b1fb-9e1ba5b1f041 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:16.756302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 384/748] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:16.761782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75dc7-c382-48d5-a48d-f799412af45f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:16.762667 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75dc7-c382-48d5-a48d-f799412af45f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:16.763155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75dc7-c382-48d5-a48d-f799412af45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:16.781963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4be75dc7-c382-48d5-a48d-f799412af45f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:16.783782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4be75dc7-c382-48d5-a48d-f799412af45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:16.852231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 365/749] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:16 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:17.121523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0c5d2-742c-44bb-b1fb-9e1ba5b1f041 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:17.154353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af6a8-eb7d-4f47-a4f6-7e2d879c2707 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.154875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af6a8-eb7d-4f47-a4f6-7e2d879c2707 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.155373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af6a8-eb7d-4f47-a4f6-7e2d879c2707 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.157199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 366/750] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:17.162341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90280c-8c71-4bbe-aea9-61ac3e341127 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.162862 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90280c-8c71-4bbe-aea9-61ac3e341127 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.163344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90280c-8c71-4bbe-aea9-61ac3e341127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.173230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:17.186791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8b90280c-8c71-4bbe-aea9-61ac3e341127 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:17.195774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8b90280c-8c71-4bbe-aea9-61ac3e341127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:17.237665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0c5d2-742c-44bb-b1fb-9e1ba5b1f041 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:17.264486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 367/751] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:17.277696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff06be7-4759-4220-9a11-08f1af4fd3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:17.278564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06be7-4759-4220-9a11-08f1af4fd3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.278987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06be7-4759-4220-9a11-08f1af4fd3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.279403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06be7-4759-4220-9a11-08f1af4fd3b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.280124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff06be7-4759-4220-9a11-08f1af4fd3b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:17.281615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff06be7-4759-4220-9a11-08f1af4fd3b1 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-6ff06be7-4759-4220-9a11-08f1af4fd3b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:17.290576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:17.291737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:21:17.292840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:17.295055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 385/752] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Oct 12 12:21:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:17.301158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90280c-8c71-4bbe-aea9-61ac3e341127 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:17.340680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:17.416269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90280c-8c71-4bbe-aea9-61ac3e341127 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:17.428080 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780de7-deab-4509-bfc2-a02e9819769e None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.428668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780de7-deab-4509-bfc2-a02e9819769e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.429141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90780de7-deab-4509-bfc2-a02e9819769e None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.431515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 386/753] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:17.436253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42a5b-e6cf-48e8-ac4b-2afdb95f2758 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.436761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42a5b-e6cf-48e8-ac4b-2afdb95f2758 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.437238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42a5b-e6cf-48e8-ac4b-2afdb95f2758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.455904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c5c42a5b-e6cf-48e8-ac4b-2afdb95f2758 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:17.456077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:17.456646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.456929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.457203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.457834 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:17.458179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:17.458653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c5c42a5b-e6cf-48e8-ac4b-2afdb95f2758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:17.459546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:17.465022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 368/754] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:17.472480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:17.545502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 387/755] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:17.552494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c42a5b-e6cf-48e8-ac4b-2afdb95f2758 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:17.590915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:17.597560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4709962-9865-4861-9228-602884c9bd1f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:17.607017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:17.640347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:17.641066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.641414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.642069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.643113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:17.643952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:17.646261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:17.654069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 369/756] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Oct 12 12:21:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:17.658782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:17.685344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c42a5b-e6cf-48e8-ac4b-2afdb95f2758 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:17.721769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:17.722731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.723214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.723962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] PATH_INFO: `/v3/services/compute` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.724763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:17.726834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:17.736307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:17.737265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:17.738136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:17.740849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 388/757] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:17.749433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:17.784659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e17e6-32fe-414a-ae83-b713d8b32fb9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:17.820999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a194713-9042-455f-a380-339ba9522adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:17.821962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194713-9042-455f-a380-339ba9522adc None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.822426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194713-9042-455f-a380-339ba9522adc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.822900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194713-9042-455f-a380-339ba9522adc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.823825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a194713-9042-455f-a380-339ba9522adc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:17.825234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a194713-9042-455f-a380-339ba9522adc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:17.859003 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b0777-6a65-46a8-89a5-639cbd1e72de None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:17.860413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:17.889998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 370/758] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:21:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:21:17.890948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a194713-9042-455f-a380-339ba9522adc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:17.908689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:17.910121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:17.910593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:17.910872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:17.911590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:17.911899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:17.913319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:17.920442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 389/759] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:21:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:17.930471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:17.931547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.049576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a194713-9042-455f-a380-339ba9522adc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.070895 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86658ad-70e0-4e70-b5b0-65f56108e424 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.099648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952c992-f8d5-4252-b791-fa601530a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.100638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c992-f8d5-4252-b791-fa601530a060 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.100987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c992-f8d5-4252-b791-fa601530a060 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.101815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952c992-f8d5-4252-b791-fa601530a060 None admin] PATH_INFO: `/v3/services/image` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.102428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c992-f8d5-4252-b791-fa601530a060 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.103790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952c992-f8d5-4252-b791-fa601530a060 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.110069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.110614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.110896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.111170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.111801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:18.112112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.113482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1952c992-f8d5-4252-b791-fa601530a060 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:18.114298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:18.115245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:18.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 371/760] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Oct 12 12:21:17 2022] GET /identity/v3/services/image => generated 86 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:18.116910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc372-b882-4a48-8d2c-6728201c37e0 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.117464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc372-b882-4a48-8d2c-6728201c37e0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.117464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc372-b882-4a48-8d2c-6728201c37e0 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.122007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 390/761] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:21:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:18.122628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 372/762] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:18.124437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.124954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bc372-b882-4a48-8d2c-6728201c37e0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.161153 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:18.218512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:18.235992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e29029-c76e-49cf-8d45-0356a7afd845 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.238201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bc372-b882-4a48-8d2c-6728201c37e0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.271975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.272561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.272830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.273129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.273747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] RBAC: Adding query filter params (name=image) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:18.274088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.275192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9376d08-13a6-4307-bfb2-6ba9e55d6a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.275425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.275799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376d08-13a6-4307-bfb2-6ba9e55d6a5a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.276082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376d08-13a6-4307-bfb2-6ba9e55d6a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.276367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376d08-13a6-4307-bfb2-6ba9e55d6a5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.277066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9376d08-13a6-4307-bfb2-6ba9e55d6a5a None admin] RBAC: Adding query filter params (interface=public, service_id=0fc49a11857844cb916369553c643761, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:18.277380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9376d08-13a6-4307-bfb2-6ba9e55d6a5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.278757 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9376d08-13a6-4307-bfb2-6ba9e55d6a5a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.280699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 391/763] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:18.285268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 373/764] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Oct 12 12:21:18 2022] GET /identity/v3/endpoints?service_id=0fc49a11857844cb916369553c643761&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:18.285476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4f29e-808d-44d2-8c5c-bec32c531490 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.285773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4f29e-808d-44d2-8c5c-bec32c531490 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.286082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4f29e-808d-44d2-8c5c-bec32c531490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.286827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.306347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b3f4f29e-808d-44d2-8c5c-bec32c531490 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:18.307547 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b3f4f29e-808d-44d2-8c5c-bec32c531490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:18.384453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:18.386141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 374/765] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:18 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:18.387017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4f29e-808d-44d2-8c5c-bec32c531490 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.412086 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a4009-0c86-4830-ac0e-18041a998d62 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.440753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.441860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.442311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.442680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.443456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] RBAC: Adding query filter params (type=image) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:18.443860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.445976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.455999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 392/766] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:18.457683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.484216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:18.516540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4f29e-808d-44d2-8c5c-bec32c531490 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.555465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09204d7c-117c-46a5-bef1-d6e90cb5659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.556189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09204d7c-117c-46a5-bef1-d6e90cb5659b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.556460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09204d7c-117c-46a5-bef1-d6e90cb5659b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.556739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09204d7c-117c-46a5-bef1-d6e90cb5659b None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.557345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09204d7c-117c-46a5-bef1-d6e90cb5659b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.558883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09204d7c-117c-46a5-bef1-d6e90cb5659b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.563716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d7231-36ee-4620-86a1-6d68fe322f6e None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.566837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 375/767] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:21:18 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:21:18.603256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.604256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.604734 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.605202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.606191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] RBAC: Adding query filter params (interface=public, service_id=95aa97f6e23349208f4a49f129a3b456, region_id=RegionOne) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:18.606740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.608318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.614792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 393/768] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Oct 12 12:21:18 2022] GET /identity/v3/endpoints?service_id=95aa97f6e23349208f4a49f129a3b456&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:18.621661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09204d7c-117c-46a5-bef1-d6e90cb5659b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.704697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.727418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:18.736137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09204d7c-117c-46a5-bef1-d6e90cb5659b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.776242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedc2ac-6316-4f58-afdb-ea1799564d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.776607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedc2ac-6316-4f58-afdb-ea1799564d03 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.776888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedc2ac-6316-4f58-afdb-ea1799564d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.777160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedc2ac-6316-4f58-afdb-ea1799564d03 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.778514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedc2ac-6316-4f58-afdb-ea1799564d03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.779165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedc2ac-6316-4f58-afdb-ea1799564d03 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.787993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 376/769] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:21:18 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:21:18.831827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3727d-9ccb-4e7d-9990-0ac7fa9f5f35 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:18.870898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:18.871530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:18.871767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:18.872037 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:18.872772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:18.874256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:18.876881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING keystone.server.flask.common [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 12 12:21:18.930045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.sql.core [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 12:21:18.930045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 12 12:21:18.930045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [parameters: {'user_id': 'c0b64d35e373444c84656889310ab2aa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 12 12:21:18.930045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81490) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result.read() Oct 12 12:21:18.937900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:21:18.938836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application yield resource Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise self.value Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application session.commit() Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:21:18.939823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:21:18.940689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:21:18.941536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result.read() Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:21:18.943844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c0b64d35e373444c84656889310ab2aa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:18.945862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 12 12:21:18.947731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 12 12:21:18.949827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 12 12:21:18.949827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 12 12:21:18.949827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:21:18.949827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:21:18.949827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 12 12:21:18.949827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:18.949827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 394/770] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Oct 12 12:21:18 2022] POST /identity/v3/users => generated 160 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:18.957220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedc2ac-6316-4f58-afdb-ea1799564d03 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:18.961376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:19.068013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedc2ac-6316-4f58-afdb-ea1799564d03 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:19.103489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161cdab-f4cb-4852-825e-1931c84681d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:19.104122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161cdab-f4cb-4852-825e-1931c84681d2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:19.104400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161cdab-f4cb-4852-825e-1931c84681d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:19.104726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161cdab-f4cb-4852-825e-1931c84681d2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:19.112624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161cdab-f4cb-4852-825e-1931c84681d2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:19.114076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161cdab-f4cb-4852-825e-1931c84681d2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-d161cdab-f4cb-4852-825e-1931c84681d2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:19.126080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:19.127037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:19.128099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 377/771] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:21:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:19.133501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:19.227425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:19.247352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f12240-2061-47a0-bef5-60bf69a4b4c8 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:19.282821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946307c-11cf-4aef-af99-b656a309991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:19.283716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946307c-11cf-4aef-af99-b656a309991a None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:19.284187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946307c-11cf-4aef-af99-b656a309991a None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:19.284620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946307c-11cf-4aef-af99-b656a309991a None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:19.285495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946307c-11cf-4aef-af99-b656a309991a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:19.286050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946307c-11cf-4aef-af99-b656a309991a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:19.287865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946307c-11cf-4aef-af99-b656a309991a None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:19.326790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 395/772] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Oct 12 12:21:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:20.099194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2c54c-e742-42d7-bbed-f81308353db0 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.099693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2c54c-e742-42d7-bbed-f81308353db0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.099767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2c54c-e742-42d7-bbed-f81308353db0 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.101293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 378/773] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:20.106747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aed78-deb5-4911-8e0a-2e50da0b00fd None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.107160 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aed78-deb5-4911-8e0a-2e50da0b00fd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.107323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aed78-deb5-4911-8e0a-2e50da0b00fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.125913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-122aed78-deb5-4911-8e0a-2e50da0b00fd None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:20.127490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-122aed78-deb5-4911-8e0a-2e50da0b00fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:20.195617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 396/774] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:20.462497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2c54c-e742-42d7-bbed-f81308353db0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:20.526475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:20.566987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2c54c-e742-42d7-bbed-f81308353db0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:20.600280 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f89019-c93c-40e3-867d-dfdb0d4fefa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:20.600936 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89019-c93c-40e3-867d-dfdb0d4fefa9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.601721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89019-c93c-40e3-867d-dfdb0d4fefa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.602026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89019-c93c-40e3-867d-dfdb0d4fefa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.603118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f89019-c93c-40e3-867d-dfdb0d4fefa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:20.604603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f89019-c93c-40e3-867d-dfdb0d4fefa9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:20.606347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 379/775] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:20.611849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122aed78-deb5-4911-8e0a-2e50da0b00fd None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:20.677654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:20.715066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122aed78-deb5-4911-8e0a-2e50da0b00fd None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:20.751387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ae561-0d5b-4e42-af11-52c5c640e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:20.751984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ae561-0d5b-4e42-af11-52c5c640e5e4 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.752259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ae561-0d5b-4e42-af11-52c5c640e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.752526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ae561-0d5b-4e42-af11-52c5c640e5e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.753174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ae561-0d5b-4e42-af11-52c5c640e5e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:20.754621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ae561-0d5b-4e42-af11-52c5c640e5e4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:20.821295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120df943-31d2-4112-a1f6-6505a325b75a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.821602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120df943-31d2-4112-a1f6-6505a325b75a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.821884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120df943-31d2-4112-a1f6-6505a325b75a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.823364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 380/776] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:20.828277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17bf7-a859-4d88-a4a9-56524e92d271 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.828560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17bf7-a859-4d88-a4a9-56524e92d271 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.828876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f17bf7-a859-4d88-a4a9-56524e92d271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.847579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-69f17bf7-a859-4d88-a4a9-56524e92d271 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:20.849225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-69f17bf7-a859-4d88-a4a9-56524e92d271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:20.855626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 397/777] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:20 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:20.928402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 381/778] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:21.053023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc9f0e-bc01-4aa2-8028-8db1a36481ec None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.053309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc9f0e-bc01-4aa2-8028-8db1a36481ec None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.053581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc9f0e-bc01-4aa2-8028-8db1a36481ec None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.055129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 398/779] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:21.060472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7076-bb56-42c2-b03c-fd15bcf19a02 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.060765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7076-bb56-42c2-b03c-fd15bcf19a02 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.061041 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7076-bb56-42c2-b03c-fd15bcf19a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.081653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-870d7076-bb56-42c2-b03c-fd15bcf19a02 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:21.082970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-870d7076-bb56-42c2-b03c-fd15bcf19a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:21.144490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:21.155397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 382/780] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:21 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:21.200258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc9f0e-bc01-4aa2-8028-8db1a36481ec None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:21.302792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc9f0e-bc01-4aa2-8028-8db1a36481ec None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:21.324134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:21.339326 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:21.339948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.340220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.340485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.341057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:21.342455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:21.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:21.358680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:21.359519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 399/781] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:21.361635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d7076-bb56-42c2-b03c-fd15bcf19a02 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:21.420976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:21.469630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d7076-bb56-42c2-b03c-fd15bcf19a02 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:21.506906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:21.508049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.508327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.508595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.509376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:21.509707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:21.511319 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:21.516669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 383/782] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:21:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:21.522110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:21.529259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a215dd0-ee1b-4473-8b9f-81030eca1415 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:21.563392 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:21.564020 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.564297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.564567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] PATH_INFO: `/v3/services/image` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.565244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:21.566613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:21.575814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:21.576817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:21.577702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 400/783] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Oct 12 12:21:21 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:21.581617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:21.582541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:21.628549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab184d34-314e-4a97-ae73-de39fe26aff6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:21.633089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:21.665585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:21.666356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.666687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.666956 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.667631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:21.667935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:21.669285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:21.679103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 384/784] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Oct 12 12:21:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:21.680447 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:21.689148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d28aa-808f-45d5-96ac-65be7ecc2384 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:21.725641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:21.726430 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.726719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.726986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.727617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:21.727929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:21.729286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:21.737981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 401/785] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:21.739900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:21.787082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe795a-4c6b-48cb-ae9e-3203b92eb619 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:21.811802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:21.829028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d2bf6-3837-4040-8817-4cd9f9784c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:21.829692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d2bf6-3837-4040-8817-4cd9f9784c43 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.829998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d2bf6-3837-4040-8817-4cd9f9784c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.830278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d2bf6-3837-4040-8817-4cd9f9784c43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.830940 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d2bf6-3837-4040-8817-4cd9f9784c43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:21.832501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d2bf6-3837-4040-8817-4cd9f9784c43 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:21.844163 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8d45-8a37-4005-8a8f-9900a8358bd0 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:21.857850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:21.878691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:21.879300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:21.879581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:21.879861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:21.880548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:21.880854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:21.882256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:21.887613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 402/786] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:21.893324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:21.955739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 385/787] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:21:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:22.006391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ad667-0033-417e-a9ab-a50bd68e86b0 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:22.050932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1258185-be58-41d1-8c48-ff6cc38adf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:22.051553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1258185-be58-41d1-8c48-ff6cc38adf04 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:22.051832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1258185-be58-41d1-8c48-ff6cc38adf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:22.052104 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1258185-be58-41d1-8c48-ff6cc38adf04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:22.052776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1258185-be58-41d1-8c48-ff6cc38adf04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:22.054221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1258185-be58-41d1-8c48-ff6cc38adf04 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:22.153681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 403/788] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:21:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:21:23.068414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968171-771a-4abc-9c2d-1a6df8eb0363 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:23.068930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968171-771a-4abc-9c2d-1a6df8eb0363 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:23.068930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21968171-771a-4abc-9c2d-1a6df8eb0363 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:23.071567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 386/789] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:23.077118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2dcd2-4a80-4435-bcb1-09f3e5486eb5 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:23.077516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2dcd2-4a80-4435-bcb1-09f3e5486eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:23.077699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2dcd2-4a80-4435-bcb1-09f3e5486eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:23.095818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-fce2dcd2-4a80-4435-bcb1-09f3e5486eb5 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:23.096994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fce2dcd2-4a80-4435-bcb1-09f3e5486eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:23.120312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:23.170697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 404/790] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:23.446645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21968171-771a-4abc-9c2d-1a6df8eb0363 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:23.549557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21968171-771a-4abc-9c2d-1a6df8eb0363 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:23.582699 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1f6e2-44a9-4a5f-9888-9da7549d66de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:23.583321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1f6e2-44a9-4a5f-9888-9da7549d66de None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:23.583604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1f6e2-44a9-4a5f-9888-9da7549d66de None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:23.583876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1f6e2-44a9-4a5f-9888-9da7549d66de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:23.584991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1f6e2-44a9-4a5f-9888-9da7549d66de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:23.586473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1f6e2-44a9-4a5f-9888-9da7549d66de None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:23.588313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 387/791] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:23.593822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2dcd2-4a80-4435-bcb1-09f3e5486eb5 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:23.694652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2dcd2-4a80-4435-bcb1-09f3e5486eb5 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:23.729365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab9a30-86e2-4a46-b44c-db1a4cbc22ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:23.729900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab9a30-86e2-4a46-b44c-db1a4cbc22ca None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:23.730206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab9a30-86e2-4a46-b44c-db1a4cbc22ca None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:23.730475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab9a30-86e2-4a46-b44c-db1a4cbc22ca None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:23.731141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab9a30-86e2-4a46-b44c-db1a4cbc22ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:23.732627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab9a30-86e2-4a46-b44c-db1a4cbc22ca None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:23.837967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 405/792] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:21:23 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:24.202107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52b01-ed0a-45e7-9131-82afe0737b19 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.202646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52b01-ed0a-45e7-9131-82afe0737b19 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.202646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52b01-ed0a-45e7-9131-82afe0737b19 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.204139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 388/793] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:24.209251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060d63-fcec-465a-bdea-bc6325f7bd22 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.209669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060d63-fcec-465a-bdea-bc6325f7bd22 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.209835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060d63-fcec-465a-bdea-bc6325f7bd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.230243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-39060d63-fcec-465a-bdea-bc6325f7bd22 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:24.231523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-39060d63-fcec-465a-bdea-bc6325f7bd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:24.234889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:24.301881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 406/794] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:24.343827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0628788-7735-4c2c-b047-9fd20de6c77f None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.344111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0628788-7735-4c2c-b047-9fd20de6c77f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.344298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0628788-7735-4c2c-b047-9fd20de6c77f None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.345802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 389/795] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:24.351356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848df37-0221-46ac-9563-174f9c209fc3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.351646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848df37-0221-46ac-9563-174f9c209fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.351909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848df37-0221-46ac-9563-174f9c209fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.369859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2848df37-0221-46ac-9563-174f9c209fc3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:24.371066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2848df37-0221-46ac-9563-174f9c209fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:24.439374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 407/796] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:24.578973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0628788-7735-4c2c-b047-9fd20de6c77f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.626979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:24.683317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0628788-7735-4c2c-b047-9fd20de6c77f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:24.709746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848df37-0221-46ac-9563-174f9c209fc3 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.718247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dbfb1-318b-47ba-a67a-0d046f96a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:24.718877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dbfb1-318b-47ba-a67a-0d046f96a8da None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.719156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dbfb1-318b-47ba-a67a-0d046f96a8da None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.719430 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dbfb1-318b-47ba-a67a-0d046f96a8da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.730992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dbfb1-318b-47ba-a67a-0d046f96a8da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-0c1dbfb1-318b-47ba-a67a-0d046f96a8da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:24.736826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:24.738088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:24.738945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 390/797] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:24.744736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dbfb1-318b-47ba-a67a-0d046f96a8da None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.786323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:24.817133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848df37-0221-46ac-9563-174f9c209fc3 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:24.848517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dbfb1-318b-47ba-a67a-0d046f96a8da None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:24.851861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:24.853841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:24.854471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.854749 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.855020 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.855691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:24.857476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:24.882530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:24.883073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.883353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.883623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.884278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:24.884578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:24.885923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:24.893710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 391/798] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:24.897151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.sql.core [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:21:24.897151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:21:24.897151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [parameters: {'id': '07c319129bad4f80a455c7cc4b2a7ecc', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 12:21:24.897151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81490) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:21:24.899450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:21:24.904854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result.read() Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 12:21:24.905688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application yield resource Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise self.value Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application session.commit() Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:21:24.906654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:21:24.908157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:21:24.909016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application result.read() Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:21:24.911348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [parameters: {'id': '07c319129bad4f80a455c7cc4b2a7ecc', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 12:21:24.913420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:24.915942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 408/799] 10.222.0.44 () {60 vars in 1137 bytes} [Wed Oct 12 12:21:24 2022] POST /identity/v3/domains => generated 190 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:24.918404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.006463 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d3309-9e9a-437d-ab60-9e90ec1dab80 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.018798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79fabc-2eb3-4d80-8ab5-e3d91d6ee699 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.041313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.042057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.042431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.042782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.053579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.055749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.057183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.057919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.058332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.058632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.060587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:25.070980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:25.077626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:25.078868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:25.078868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:25.078868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:25.078868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 392/800] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:25.078868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.083225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.084717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.099046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:25.101887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:25.102928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:25.103809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:25.103809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:25.103809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:25.103809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 409/801] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:25.108906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.189058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc1005-0f41-422e-9b34-c9137d4ee19a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.214247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44deb08-10ca-40a5-a9c6-6f0d2fc8698e None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.229783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.230721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.231002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.231276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.231918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:25.232225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.233596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.245975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 393/802] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:25.250807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.251358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.251463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.251721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.251948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.252613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:25.252916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.254301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.273977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 410/803] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:25.345354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.369127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4795ab4-7a18-4b4e-87ea-70224c326099 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.408314 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.408966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.409249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.409548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.418531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.420085 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:25.432918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:25.434047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:25.435510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 394/804] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:25.440072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.490480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.551411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0b00c-8a78-4be7-a2e4-b77968660d51 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.587829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.588429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.588702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.588970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.589705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:25.590075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.591484 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.610240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 411/805] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:25.616173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.714038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb502c-200a-4d52-b2d5-01c4fdbe98e3 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.745446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.746083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.746361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.746625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.753365 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.757054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.759004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:25.772612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:25.774323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:25.775209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:25.775209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:25.775209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:25.778961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 395/806] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:25.780179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.877653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dda58-9c0e-472f-8077-24c0652dda73 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.880685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.912505 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13322e-e2f1-461e-ba88-40024e0c10c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.913125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13322e-e2f1-461e-ba88-40024e0c10c4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.913400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13322e-e2f1-461e-ba88-40024e0c10c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.913715 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13322e-e2f1-461e-ba88-40024e0c10c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.914401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13322e-e2f1-461e-ba88-40024e0c10c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:25.914707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13322e-e2f1-461e-ba88-40024e0c10c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.916112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13322e-e2f1-461e-ba88-40024e0c10c4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.928793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 412/807] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:25.934705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:26.030969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706abb33-9d0d-4f5c-b112-3a78c24f1603 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:26.031184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706abb33-9d0d-4f5c-b112-3a78c24f1603 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:26.031438 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706abb33-9d0d-4f5c-b112-3a78c24f1603 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:26.032765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 413/808] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:26.034354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ae3dc-baa8-4b3c-b778-59e8cc335881 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:26.037447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f332-9a74-406d-9301-52f1905bda48 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:26.037757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f332-9a74-406d-9301-52f1905bda48 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:26.038065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7f332-9a74-406d-9301-52f1905bda48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:26.056511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c3f7f332-9a74-406d-9301-52f1905bda48 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:26.057682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c3f7f332-9a74-406d-9301-52f1905bda48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:26.070772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:26.071462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:26.071797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:26.072070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:26.078864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:26.080296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:26.091344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:26.092229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:26.093295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 396/809] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:26.098078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:26.127019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 414/810] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:26.147780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:26.194693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ab84e-f57b-40dd-b55f-7bda1fcb04cc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:26.227797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:26.228410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:26.228670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:26.228942 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:26.229622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:26.229927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:26.231356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:26.243371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 397/811] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:26.249318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7f332-9a74-406d-9301-52f1905bda48 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:26.339674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:26.346313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7f332-9a74-406d-9301-52f1905bda48 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:26.380691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0bfc1-36b8-42ff-b9fa-7300a5c4d0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:26.381347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0bfc1-36b8-42ff-b9fa-7300a5c4d0f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:26.381627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0bfc1-36b8-42ff-b9fa-7300a5c4d0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:26.381901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0bfc1-36b8-42ff-b9fa-7300a5c4d0f0 None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/a0a7230c6da74e6484db3f260b4c58df/roles/390d6cb1aefc46ea875435aa9fded217` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:26.384564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0bfc1-36b8-42ff-b9fa-7300a5c4d0f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=a0a7230c6da74e6484db3f260b4c58df, role_id=390d6cb1aefc46ea875435aa9fded217)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:26.386005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0bfc1-36b8-42ff-b9fa-7300a5c4d0f0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:26.395795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:26.440174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 415/812] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:21:26 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/a0a7230c6da74e6484db3f260b4c58df/roles/390d6cb1aefc46ea875435aa9fded217 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:21:26.499826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2db0a5-73db-4b83-9d78-baaecec9d67f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:26.513023 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:26.538453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de0353-fe16-4178-8df1-9ed19b939393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:26.539093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de0353-fe16-4178-8df1-9ed19b939393 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:26.539369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de0353-fe16-4178-8df1-9ed19b939393 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:26.539636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de0353-fe16-4178-8df1-9ed19b939393 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:26.550431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de0353-fe16-4178-8df1-9ed19b939393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-86de0353-fe16-4178-8df1-9ed19b939393 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:26.555291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:26.556393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:26.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:26.558833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 398/813] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:21:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:26.562635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0bfc1-36b8-42ff-b9fa-7300a5c4d0f0 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:26.664334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0bfc1-36b8-42ff-b9fa-7300a5c4d0f0 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:26.697291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:26.697896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:26.698195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:26.698467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:26.699126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:26.699426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:26.700778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:26.708727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 416/814] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:21:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:26.715097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de0353-fe16-4178-8df1-9ed19b939393 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:26.811319 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de0353-fe16-4178-8df1-9ed19b939393 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:26.843353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f29f70-4a7f-440a-a42d-3a44ee1eb478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:26.843965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29f70-4a7f-440a-a42d-3a44ee1eb478 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:26.844238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29f70-4a7f-440a-a42d-3a44ee1eb478 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:26.844503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29f70-4a7f-440a-a42d-3a44ee1eb478 None admin] PATH_INFO: `/v3/users/209c3dc551ed4b329abbc1dd466cf211` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:26.857803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f29f70-4a7f-440a-a42d-3a44ee1eb478 None admin] RBAC: Authorizing `identity:get_user(user_id=209c3dc551ed4b329abbc1dd466cf211)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:26.859278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f29f70-4a7f-440a-a42d-3a44ee1eb478 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:26.860903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 399/815] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:21:26 2022] GET /identity/v3/users/209c3dc551ed4b329abbc1dd466cf211 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:26.866641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:26.886162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:26.966071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d22ca-a63a-48f0-9b08-5c3df5b6f85f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:26.999491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d020dc4-df4f-4cf9-a2da-adf8d725e851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:27.000113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d020dc4-df4f-4cf9-a2da-adf8d725e851 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:27.000393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d020dc4-df4f-4cf9-a2da-adf8d725e851 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:27.000669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d020dc4-df4f-4cf9-a2da-adf8d725e851 None admin] PATH_INFO: `/v3/projects/5f367da6415345a5b232dd14b6cd53c7` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:27.001776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d020dc4-df4f-4cf9-a2da-adf8d725e851 None admin] RBAC: Authorizing `identity:get_project(project_id=5f367da6415345a5b232dd14b6cd53c7)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:27.003222 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d020dc4-df4f-4cf9-a2da-adf8d725e851 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:27.004855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 417/816] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:21:26 2022] GET /identity/v3/projects/5f367da6415345a5b232dd14b6cd53c7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:27.011307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f29f70-4a7f-440a-a42d-3a44ee1eb478 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:27.038415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:27.112159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f29f70-4a7f-440a-a42d-3a44ee1eb478 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:27.145543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc01a3-28fa-409f-9022-3dd896b03f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:27.146238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc01a3-28fa-409f-9022-3dd896b03f67 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:27.146522 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc01a3-28fa-409f-9022-3dd896b03f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:27.146799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc01a3-28fa-409f-9022-3dd896b03f67 None admin] PATH_INFO: `/v3/projects/5f367da6415345a5b232dd14b6cd53c7/users/209c3dc551ed4b329abbc1dd466cf211/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:27.148719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc01a3-28fa-409f-9022-3dd896b03f67 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f367da6415345a5b232dd14b6cd53c7, user_id=209c3dc551ed4b329abbc1dd466cf211, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:27.150174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc01a3-28fa-409f-9022-3dd896b03f67 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:27.232626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 400/817] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:21:27 2022] PUT /identity/v3/projects/5f367da6415345a5b232dd14b6cd53c7/users/209c3dc551ed4b329abbc1dd466cf211/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:21:28.558967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413a9e1-bfd7-4ec2-9c90-8751b484502e None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:28.559482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413a9e1-bfd7-4ec2-9c90-8751b484502e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:28.559568 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413a9e1-bfd7-4ec2-9c90-8751b484502e None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:28.560874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 418/818] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:28.566198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915288a1-7789-40e9-ac4d-b488e57c961e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:28.566611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915288a1-7789-40e9-ac4d-b488e57c961e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:28.566751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915288a1-7789-40e9-ac4d-b488e57c961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:28.584946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-915288a1-7789-40e9-ac4d-b488e57c961e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:28.586178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-915288a1-7789-40e9-ac4d-b488e57c961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:28.650458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 401/819] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:28.914574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413a9e1-bfd7-4ec2-9c90-8751b484502e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:28.997996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.015264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413a9e1-bfd7-4ec2-9c90-8751b484502e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.048509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86946964-d76e-4dc8-aa2b-5383d0e57cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.049126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946964-d76e-4dc8-aa2b-5383d0e57cda None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.049400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946964-d76e-4dc8-aa2b-5383d0e57cda None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.049667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86946964-d76e-4dc8-aa2b-5383d0e57cda None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.060070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86946964-d76e-4dc8-aa2b-5383d0e57cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-86946964-d76e-4dc8-aa2b-5383d0e57cda None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:29.064835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:29.065775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:29.067906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 419/820] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Oct 12 12:21:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:29.072514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915288a1-7789-40e9-ac4d-b488e57c961e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.090560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.168578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915288a1-7789-40e9-ac4d-b488e57c961e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.208284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2a2f8-8c80-456d-a747-16b31dce9e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.208965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a2f8-8c80-456d-a747-16b31dce9e2a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.209313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a2f8-8c80-456d-a747-16b31dce9e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.209575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2a2f8-8c80-456d-a747-16b31dce9e2a None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.210352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2a2f8-8c80-456d-a747-16b31dce9e2a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:29.210717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2a2f8-8c80-456d-a747-16b31dce9e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:29.212080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2a2f8-8c80-456d-a747-16b31dce9e2a None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:29.220049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 402/821] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:29.226404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86946964-d76e-4dc8-aa2b-5383d0e57cda None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.329655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86946964-d76e-4dc8-aa2b-5383d0e57cda None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.355931 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d934-46e4-4879-b121-2818e87cc49e None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.356221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d934-46e4-4879-b121-2818e87cc49e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.356557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d934-46e4-4879-b121-2818e87cc49e None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.357889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 403/822] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:29.362878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4bf42-a225-4b7f-ad63-e920c6871da9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.363158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4bf42-a225-4b7f-ad63-e920c6871da9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.363516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4bf42-a225-4b7f-ad63-e920c6871da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.364287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.364914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.365193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.365462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.376404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:29.377842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:29.380360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.382168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c4d4bf42-a225-4b7f-ad63-e920c6871da9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:29.383347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c4d4bf42-a225-4b7f-ad63-e920c6871da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:29.392865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:29.395478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:29.396647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:29.396647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:29.396647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:29.396647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 420/823] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:29.399620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.458106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 404/824] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:29.677328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78678b-7c6f-406d-9584-48d4a65391a3 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.710148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.710683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.710959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.711227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.711857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:29.712158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:29.713504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:29.719203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4bf42-a225-4b7f-ad63-e920c6871da9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.725305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 421/825] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:29.731216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.732462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.786217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.820682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4bf42-a225-4b7f-ad63-e920c6871da9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.829531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cdebb-ea7f-4657-a9c7-21aff194cf0c None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.854744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f2c90-663f-45d5-877f-f128dd445bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.855365 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f2c90-663f-45d5-877f-f128dd445bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.855640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f2c90-663f-45d5-877f-f128dd445bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.855911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f2c90-663f-45d5-877f-f128dd445bf5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.863124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.863731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.864008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.864276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.865576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f2c90-663f-45d5-877f-f128dd445bf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-1c2f2c90-663f-45d5-877f-f128dd445bf5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:29.870448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:29.871369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:29.872201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:29.872855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:29.872924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 405/826] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:29.872997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:29.877372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f2c90-663f-45d5-877f-f128dd445bf5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:29.883821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:29.884902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:29.885853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 422/827] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:29.890320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.946227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.976914 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f2c90-663f-45d5-877f-f128dd445bf5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.990502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b4440-d4dc-4ffc-bd01-66b9f32c1141 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.005958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.010057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.010648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.010922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.011197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.011851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:30.012144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.013510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.021512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 406/828] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:30.023799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.024400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.024677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.024949 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.025667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:30.026001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.027411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.027529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.045487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 423/829] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:30.051051 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.129876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f964e6f-3928-4c3f-8699-a4eee33624b5 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.150474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b6543-cf42-4607-aa65-7f18eecf9083 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.159960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.164800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f00d1-dea2-4290-8cf7-09d6f6a45fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.165424 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f00d1-dea2-4290-8cf7-09d6f6a45fcc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.165704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f00d1-dea2-4290-8cf7-09d6f6a45fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.166013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f00d1-dea2-4290-8cf7-09d6f6a45fcc None admin] PATH_INFO: `/v3/users/209c3dc551ed4b329abbc1dd466cf211` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.167144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f00d1-dea2-4290-8cf7-09d6f6a45fcc None admin] RBAC: Authorizing `identity:get_user(user_id=209c3dc551ed4b329abbc1dd466cf211)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.169783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f00d1-dea2-4290-8cf7-09d6f6a45fcc None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.170149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 407/830] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/users/209c3dc551ed4b329abbc1dd466cf211 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:30.175721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f00d1-dea2-4290-8cf7-09d6f6a45fcc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.183991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246929e-a524-4876-a8e6-eca9a72c346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.184602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246929e-a524-4876-a8e6-eca9a72c346b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.184880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246929e-a524-4876-a8e6-eca9a72c346b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.185151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246929e-a524-4876-a8e6-eca9a72c346b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.228583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.252735 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246929e-a524-4876-a8e6-eca9a72c346b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.254177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246929e-a524-4876-a8e6-eca9a72c346b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.275433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f00d1-dea2-4290-8cf7-09d6f6a45fcc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.307241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f7999-f587-4369-87a4-bcb26ed45e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.307833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f7999-f587-4369-87a4-bcb26ed45e8e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.308104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f7999-f587-4369-87a4-bcb26ed45e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.308525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f7999-f587-4369-87a4-bcb26ed45e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.352951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 424/831] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 12 12:21:30.358721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246929e-a524-4876-a8e6-eca9a72c346b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.372861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.375367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f7999-f587-4369-87a4-bcb26ed45e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.376791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f7999-f587-4369-87a4-bcb26ed45e8e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.431104 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.455467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246929e-a524-4876-a8e6-eca9a72c346b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.472858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 408/832] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 12 12:21:30.478895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f7999-f587-4369-87a4-bcb26ed45e8e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.489464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.490146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.490446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.490716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.501504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.502947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:30.517206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:30.518284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:30.519179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:30.519179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:30.519179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:30.519179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 425/833] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:30.524114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.575129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.578044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f7999-f587-4369-87a4-bcb26ed45e8e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.624091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad390ea-345c-4eb3-9157-2e09c3e6ff6e None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.651607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.660242 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.664593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.664593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.664593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.664593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:30.664593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.664593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.676602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 426/834] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:30.682944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.726308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010856d-a515-49bc-b321-b8e3494d4d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.726901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010856d-a515-49bc-b321-b8e3494d4d38 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.727177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010856d-a515-49bc-b321-b8e3494d4d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.727444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010856d-a515-49bc-b321-b8e3494d4d38 None admin] PATH_INFO: `/v3/projects/5f367da6415345a5b232dd14b6cd53c7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.728526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010856d-a515-49bc-b321-b8e3494d4d38 None admin] RBAC: Authorizing `identity:get_project(project_id=5f367da6415345a5b232dd14b6cd53c7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.729999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010856d-a515-49bc-b321-b8e3494d4d38 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.731538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 409/835] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/projects/5f367da6415345a5b232dd14b6cd53c7 => generated 311 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:30.737359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010856d-a515-49bc-b321-b8e3494d4d38 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.782026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab180925-9c4a-495b-b9c1-69d1bce522c6 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.815672 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.816296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.816576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.816847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.824015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.825443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.835358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010856d-a515-49bc-b321-b8e3494d4d38 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:30.836396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:30.837330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:30.838273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 427/836] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:30.843345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.870783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.871377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.871649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.871923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.872637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=5f367da6415345a5b232dd14b6cd53c7, user.id=209c3dc551ed4b329abbc1dd466cf211) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:30.872940 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.874368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.880656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 410/837] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/role_assignments?user.id=209c3dc551ed4b329abbc1dd466cf211&scope.project.id=5f367da6415345a5b232dd14b6cd53c7&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:30.889778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.948310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf702b0-32ed-4e6a-9b00-b4c7bf220234 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.983874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a78a34-8765-41f4-94b8-78fb816a73ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.984407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a78a34-8765-41f4-94b8-78fb816a73ce None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.984683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a78a34-8765-41f4-94b8-78fb816a73ce None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.984947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a78a34-8765-41f4-94b8-78fb816a73ce None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.985651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a78a34-8765-41f4-94b8-78fb816a73ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:30.985982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a78a34-8765-41f4-94b8-78fb816a73ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.987412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a78a34-8765-41f4-94b8-78fb816a73ce None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:31.001261 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 428/838] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:31.007513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:31.040576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:31.110788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2e24f-55bf-45f5-9b92-e5fcae3db041 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:31.143502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e4c03-d7b0-4863-9217-70ab490a488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:31.144096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e4c03-d7b0-4863-9217-70ab490a488c None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:31.144370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e4c03-d7b0-4863-9217-70ab490a488c None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:31.144637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e4c03-d7b0-4863-9217-70ab490a488c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:31.145348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e4c03-d7b0-4863-9217-70ab490a488c None admin] RBAC: Adding query filter params (role.id=390d6cb1aefc46ea875435aa9fded217, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=a0a7230c6da74e6484db3f260b4c58df) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:31.145654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e4c03-d7b0-4863-9217-70ab490a488c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:31.147092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e4c03-d7b0-4863-9217-70ab490a488c None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:31.152691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 411/839] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:21:31 2022] GET /identity/v3/role_assignments?user.id=a0a7230c6da74e6484db3f260b4c58df&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=390d6cb1aefc46ea875435aa9fded217 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:33.016578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e563bd-a3ce-47e1-83bc-90185b1e542c None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:33.017106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e563bd-a3ce-47e1-83bc-90185b1e542c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:33.017106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e563bd-a3ce-47e1-83bc-90185b1e542c None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:33.018473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 429/840] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:33.023702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77bf4d-9006-4ed3-8455-06194df23995 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:33.024069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77bf4d-9006-4ed3-8455-06194df23995 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:33.024247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77bf4d-9006-4ed3-8455-06194df23995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:33.041827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9d77bf4d-9006-4ed3-8455-06194df23995 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:33.043029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9d77bf4d-9006-4ed3-8455-06194df23995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:33.107605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 412/841] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:33.376308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e563bd-a3ce-47e1-83bc-90185b1e542c None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:33.473497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e563bd-a3ce-47e1-83bc-90185b1e542c None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:33.481318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:33.506959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ee6b8-8283-4be9-9b41-f08e20ad9b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:33.507562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee6b8-8283-4be9-9b41-f08e20ad9b62 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:33.507868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee6b8-8283-4be9-9b41-f08e20ad9b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:33.508129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ee6b8-8283-4be9-9b41-f08e20ad9b62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:33.509480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ee6b8-8283-4be9-9b41-f08e20ad9b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:33.511211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ee6b8-8283-4be9-9b41-f08e20ad9b62 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:33.512745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 430/842] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:33.518401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77bf4d-9006-4ed3-8455-06194df23995 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:33.544795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:33.612909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77bf4d-9006-4ed3-8455-06194df23995 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:33.644665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba730a-6e43-4837-a9d5-c0ef77752edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:33.645195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba730a-6e43-4837-a9d5-c0ef77752edd None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:33.645471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba730a-6e43-4837-a9d5-c0ef77752edd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:33.645738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba730a-6e43-4837-a9d5-c0ef77752edd None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:33.646413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba730a-6e43-4837-a9d5-c0ef77752edd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:33.647818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba730a-6e43-4837-a9d5-c0ef77752edd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:33.745869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 413/843] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:21:33 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:35.880986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e002a-4efd-4270-b7c6-7d34a50e7ffd None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:35.881536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e002a-4efd-4270-b7c6-7d34a50e7ffd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:35.881536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e002a-4efd-4270-b7c6-7d34a50e7ffd None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:35.882926 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 431/844] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:35.888076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b5c1d-3966-4c15-a767-b3e9526fecdc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:35.888480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b5c1d-3966-4c15-a767-b3e9526fecdc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:35.888627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b5c1d-3966-4c15-a767-b3e9526fecdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:35.906776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-af8b5c1d-3966-4c15-a767-b3e9526fecdc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:35.907945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-af8b5c1d-3966-4c15-a767-b3e9526fecdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:35.970963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 414/845] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:36.241665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e002a-4efd-4270-b7c6-7d34a50e7ffd None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:36.340272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e002a-4efd-4270-b7c6-7d34a50e7ffd None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:36.373331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f77c3-3347-41fa-94a5-7a3d6453d383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:36.373976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f77c3-3347-41fa-94a5-7a3d6453d383 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:36.374251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f77c3-3347-41fa-94a5-7a3d6453d383 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:36.374521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f77c3-3347-41fa-94a5-7a3d6453d383 None admin] PATH_INFO: `/v3/roles/5c4536b29c2a4ad2a8d05d6e94e6a186` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:36.375930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f77c3-3347-41fa-94a5-7a3d6453d383 None admin] RBAC: Authorizing `identity:get_role(role_id=5c4536b29c2a4ad2a8d05d6e94e6a186)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:36.377276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f77c3-3347-41fa-94a5-7a3d6453d383 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:36.378601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 432/846] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:21:36 2022] GET /identity/v3/roles/5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:36.385035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b5c1d-3966-4c15-a767-b3e9526fecdc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:36.399379 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:36.481767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b5c1d-3966-4c15-a767-b3e9526fecdc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:36.515727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3ec46-f9d6-4bcc-bbe2-4c511515bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:36.516292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ec46-f9d6-4bcc-bbe2-4c511515bd12 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:36.516568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ec46-f9d6-4bcc-bbe2-4c511515bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:36.516839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ec46-f9d6-4bcc-bbe2-4c511515bd12 None admin] PATH_INFO: `/v3/users/7ee9992ca4214f658ed06e39e52cf18c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:36.529487 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3ec46-f9d6-4bcc-bbe2-4c511515bd12 None admin] RBAC: Authorizing `identity:get_user(user_id=7ee9992ca4214f658ed06e39e52cf18c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:36.530930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3ec46-f9d6-4bcc-bbe2-4c511515bd12 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:36.532492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 415/847] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:21:36 2022] GET /identity/v3/users/7ee9992ca4214f658ed06e39e52cf18c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:36.538340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f77c3-3347-41fa-94a5-7a3d6453d383 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:36.581798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:36.637802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f77c3-3347-41fa-94a5-7a3d6453d383 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:36.669940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a433c6-d786-4a28-ba1f-f559c0f5c415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:36.670567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a433c6-d786-4a28-ba1f-f559c0f5c415 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:36.670844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a433c6-d786-4a28-ba1f-f559c0f5c415 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:36.671115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a433c6-d786-4a28-ba1f-f559c0f5c415 None admin] PATH_INFO: `/v3/projects/5f367da6415345a5b232dd14b6cd53c7` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:36.672210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a433c6-d786-4a28-ba1f-f559c0f5c415 None admin] RBAC: Authorizing `identity:get_project(project_id=5f367da6415345a5b232dd14b6cd53c7)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:36.673622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a433c6-d786-4a28-ba1f-f559c0f5c415 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:36.675361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 433/848] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:21:36 2022] GET /identity/v3/projects/5f367da6415345a5b232dd14b6cd53c7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:36.681738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3ec46-f9d6-4bcc-bbe2-4c511515bd12 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:36.748133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:36.777060 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3ec46-f9d6-4bcc-bbe2-4c511515bd12 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:36.810182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe88064-a708-4a19-a23f-7bfe91c222ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:36.810823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe88064-a708-4a19-a23f-7bfe91c222ec None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:36.811101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe88064-a708-4a19-a23f-7bfe91c222ec None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:36.811370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe88064-a708-4a19-a23f-7bfe91c222ec None admin] PATH_INFO: `/v3/projects/5f367da6415345a5b232dd14b6cd53c7/users/7ee9992ca4214f658ed06e39e52cf18c/roles/5c4536b29c2a4ad2a8d05d6e94e6a186` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:36.813479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe88064-a708-4a19-a23f-7bfe91c222ec None admin] RBAC: Authorizing `identity:create_grant(project_id=5f367da6415345a5b232dd14b6cd53c7, user_id=7ee9992ca4214f658ed06e39e52cf18c, role_id=5c4536b29c2a4ad2a8d05d6e94e6a186)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:36.814910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe88064-a708-4a19-a23f-7bfe91c222ec None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:36.880000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 416/849] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:21:36 2022] PUT /identity/v3/projects/5f367da6415345a5b232dd14b6cd53c7/users/7ee9992ca4214f658ed06e39e52cf18c/roles/5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:21:38.958834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d53e7-06a0-41e6-8be7-5253b72aafbe None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:38.959393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d53e7-06a0-41e6-8be7-5253b72aafbe None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:38.959393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d53e7-06a0-41e6-8be7-5253b72aafbe None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:38.961215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 434/850] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:38.966501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df66c0-374e-4cb9-9706-c17f2213d584 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:38.966918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df66c0-374e-4cb9-9706-c17f2213d584 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:38.967024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df66c0-374e-4cb9-9706-c17f2213d584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:38.985344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b7df66c0-374e-4cb9-9706-c17f2213d584 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:38.986551 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b7df66c0-374e-4cb9-9706-c17f2213d584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:39.050957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 417/851] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:39.317701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d53e7-06a0-41e6-8be7-5253b72aafbe None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.416814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d53e7-06a0-41e6-8be7-5253b72aafbe None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:39.427257 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:39.450471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91595e8-d4f2-49fd-8185-29ae7c59a4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:39.451086 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91595e8-d4f2-49fd-8185-29ae7c59a4de None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:39.451434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91595e8-d4f2-49fd-8185-29ae7c59a4de None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:39.451713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91595e8-d4f2-49fd-8185-29ae7c59a4de None admin] PATH_INFO: `/v3/roles/5c4536b29c2a4ad2a8d05d6e94e6a186` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:39.453226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91595e8-d4f2-49fd-8185-29ae7c59a4de None admin] RBAC: Authorizing `identity:get_role(role_id=5c4536b29c2a4ad2a8d05d6e94e6a186)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:39.454612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91595e8-d4f2-49fd-8185-29ae7c59a4de None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:39.455926 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 435/852] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/roles/5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:39.462283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df66c0-374e-4cb9-9706-c17f2213d584 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.519012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:39.557081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df66c0-374e-4cb9-9706-c17f2213d584 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:39.590868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c396a1-ae53-4b1e-8c23-2ba5623505fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:39.591428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c396a1-ae53-4b1e-8c23-2ba5623505fd None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:39.591707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c396a1-ae53-4b1e-8c23-2ba5623505fd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:39.591979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c396a1-ae53-4b1e-8c23-2ba5623505fd None admin] PATH_INFO: `/v3/users/7ee9992ca4214f658ed06e39e52cf18c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:39.593089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c396a1-ae53-4b1e-8c23-2ba5623505fd None admin] RBAC: Authorizing `identity:get_user(user_id=7ee9992ca4214f658ed06e39e52cf18c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:39.594532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c396a1-ae53-4b1e-8c23-2ba5623505fd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:39.596088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 418/853] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/users/7ee9992ca4214f658ed06e39e52cf18c => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:39.602141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91595e8-d4f2-49fd-8185-29ae7c59a4de None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.698996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91595e8-d4f2-49fd-8185-29ae7c59a4de None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:39.732187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3e851-140d-4d8e-a8fa-e87c646b9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:39.732768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e851-140d-4d8e-a8fa-e87c646b9647 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:39.733044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e851-140d-4d8e-a8fa-e87c646b9647 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:39.733313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e851-140d-4d8e-a8fa-e87c646b9647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:39.777186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:39.797527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3e851-140d-4d8e-a8fa-e87c646b9647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:39.798955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3e851-140d-4d8e-a8fa-e87c646b9647 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:39.894147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 436/854] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:21:39.899937 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c396a1-ae53-4b1e-8c23-2ba5623505fd None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.996809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c396a1-ae53-4b1e-8c23-2ba5623505fd None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:40.021747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:40.029196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721567a-ae72-46e3-acd0-9b8f757fa28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:40.029812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721567a-ae72-46e3-acd0-9b8f757fa28f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:40.030124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721567a-ae72-46e3-acd0-9b8f757fa28f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:40.035277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721567a-ae72-46e3-acd0-9b8f757fa28f None admin] PATH_INFO: `/v3/projects/5f367da6415345a5b232dd14b6cd53c7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:40.035277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721567a-ae72-46e3-acd0-9b8f757fa28f None admin] RBAC: Authorizing `identity:get_project(project_id=5f367da6415345a5b232dd14b6cd53c7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:40.035277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721567a-ae72-46e3-acd0-9b8f757fa28f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:40.035277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 419/855] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/projects/5f367da6415345a5b232dd14b6cd53c7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:40.041266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3e851-140d-4d8e-a8fa-e87c646b9647 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:40.117358 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:40.137829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3e851-140d-4d8e-a8fa-e87c646b9647 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:40.171486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815221b7-c51c-4b69-a20b-7823aed49f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:40.172027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815221b7-c51c-4b69-a20b-7823aed49f56 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:40.172306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815221b7-c51c-4b69-a20b-7823aed49f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:40.172576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815221b7-c51c-4b69-a20b-7823aed49f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:40.173292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815221b7-c51c-4b69-a20b-7823aed49f56 None admin] RBAC: Adding query filter params (role.id=5c4536b29c2a4ad2a8d05d6e94e6a186, scope.project.id=5f367da6415345a5b232dd14b6cd53c7, user.id=7ee9992ca4214f658ed06e39e52cf18c) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:40.173602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815221b7-c51c-4b69-a20b-7823aed49f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:40.175047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815221b7-c51c-4b69-a20b-7823aed49f56 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:40.181249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 437/856] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:21:40 2022] GET /identity/v3/role_assignments?user.id=7ee9992ca4214f658ed06e39e52cf18c&scope.project.id=5f367da6415345a5b232dd14b6cd53c7&role.id=5c4536b29c2a4ad2a8d05d6e94e6a186 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:42.308003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8b42f-a1fe-495a-a5a8-0b10a5133005 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:42.308518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8b42f-a1fe-495a-a5a8-0b10a5133005 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:42.308518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8b42f-a1fe-495a-a5a8-0b10a5133005 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:42.309869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 420/857] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:42.315045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42327b10-3053-4c2e-a2d2-48619f7dc5c3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:42.315439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42327b10-3053-4c2e-a2d2-48619f7dc5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:42.315590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42327b10-3053-4c2e-a2d2-48619f7dc5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:42.333529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-42327b10-3053-4c2e-a2d2-48619f7dc5c3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:42.334693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-42327b10-3053-4c2e-a2d2-48619f7dc5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:42.399122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 438/858] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:42.665130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8b42f-a1fe-495a-a5a8-0b10a5133005 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:42.763314 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8b42f-a1fe-495a-a5a8-0b10a5133005 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:42.795380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866c1a8-476f-4724-ad20-7989d11c9148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:42.795985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866c1a8-476f-4724-ad20-7989d11c9148 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:42.796261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866c1a8-476f-4724-ad20-7989d11c9148 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:42.796530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866c1a8-476f-4724-ad20-7989d11c9148 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:42.797626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866c1a8-476f-4724-ad20-7989d11c9148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:42.799056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866c1a8-476f-4724-ad20-7989d11c9148 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:42.800552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 421/859] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:42.806642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42327b10-3053-4c2e-a2d2-48619f7dc5c3 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:42.875154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:42.903196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42327b10-3053-4c2e-a2d2-48619f7dc5c3 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:42.936938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc14d8-5865-4f7f-91e9-2896f48f8fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:42.937472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc14d8-5865-4f7f-91e9-2896f48f8fe1 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:42.937750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc14d8-5865-4f7f-91e9-2896f48f8fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:42.938049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc14d8-5865-4f7f-91e9-2896f48f8fe1 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:42.938686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc14d8-5865-4f7f-91e9-2896f48f8fe1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:42.940084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc14d8-5865-4f7f-91e9-2896f48f8fe1 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:43.041742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 439/860] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:42 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:45.196927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b23d5a-95b4-4c35-91fb-4bc6e541c913 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:45.197440 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b23d5a-95b4-4c35-91fb-4bc6e541c913 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:45.197440 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b23d5a-95b4-4c35-91fb-4bc6e541c913 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:45.198805 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 422/861] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:45.204125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260fcbc-86da-4341-a977-226ea6c4afcc None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:45.204530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260fcbc-86da-4341-a977-226ea6c4afcc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:45.204677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260fcbc-86da-4341-a977-226ea6c4afcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:45.223132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9260fcbc-86da-4341-a977-226ea6c4afcc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:45.224593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9260fcbc-86da-4341-a977-226ea6c4afcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:45.289730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 440/862] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:45.555475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b23d5a-95b4-4c35-91fb-4bc6e541c913 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:45.615789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:45.651744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b23d5a-95b4-4c35-91fb-4bc6e541c913 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:45.684122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e57d1f-0bff-448d-aeb4-b9b6ddf64007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:45.684727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57d1f-0bff-448d-aeb4-b9b6ddf64007 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:45.685001 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57d1f-0bff-448d-aeb4-b9b6ddf64007 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:45.685264 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57d1f-0bff-448d-aeb4-b9b6ddf64007 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:45.686658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e57d1f-0bff-448d-aeb4-b9b6ddf64007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:45.688051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e57d1f-0bff-448d-aeb4-b9b6ddf64007 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:45.689550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 423/863] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:45.695203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260fcbc-86da-4341-a977-226ea6c4afcc None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:45.747912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:45.791028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260fcbc-86da-4341-a977-226ea6c4afcc None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:45.823408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5612df9-549f-452f-88d1-d877ac4d6112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:45.823935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5612df9-549f-452f-88d1-d877ac4d6112 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:45.824209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5612df9-549f-452f-88d1-d877ac4d6112 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:45.824476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5612df9-549f-452f-88d1-d877ac4d6112 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:45.825107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5612df9-549f-452f-88d1-d877ac4d6112 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:45.826570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5612df9-549f-452f-88d1-d877ac4d6112 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:45.914814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 441/864] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:21:45 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:48.043364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fea30-6e92-4c84-930f-812e9a04ade5 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.043949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fea30-6e92-4c84-930f-812e9a04ade5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.043949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fea30-6e92-4c84-930f-812e9a04ade5 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.045231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 424/865] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:48.050573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85e4ec-b5f2-4d95-a603-c035e2d20e2b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.050937 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85e4ec-b5f2-4d95-a603-c035e2d20e2b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.051115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85e4ec-b5f2-4d95-a603-c035e2d20e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.069337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-0b85e4ec-b5f2-4d95-a603-c035e2d20e2b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:48.071200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0b85e4ec-b5f2-4d95-a603-c035e2d20e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:48.135486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 442/866] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:48.398406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fea30-6e92-4c84-930f-812e9a04ade5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:48.495230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fea30-6e92-4c84-930f-812e9a04ade5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:48.522436 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:48.527626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70145bc-7f27-4f9a-98ca-a72e458409c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:48.528229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70145bc-7f27-4f9a-98ca-a72e458409c9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.528503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70145bc-7f27-4f9a-98ca-a72e458409c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.528768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70145bc-7f27-4f9a-98ca-a72e458409c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.538365 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70145bc-7f27-4f9a-98ca-a72e458409c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-f70145bc-7f27-4f9a-98ca-a72e458409c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:48.543996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:48.545067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:21:48.545923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 425/867] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:21:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:48.550996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85e4ec-b5f2-4d95-a603-c035e2d20e2b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:48.591046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:48.646476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85e4ec-b5f2-4d95-a603-c035e2d20e2b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:48.680218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:48.680751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.681023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.681290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.681958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:48.682255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:48.683597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:48.691273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 443/868] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:48.696935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70145bc-7f27-4f9a-98ca-a72e458409c9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:48.792104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70145bc-7f27-4f9a-98ca-a72e458409c9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:48.823267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfd221-04c2-4925-882a-de49f3c4ee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:48.823865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfd221-04c2-4925-882a-de49f3c4ee37 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.824137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfd221-04c2-4925-882a-de49f3c4ee37 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.824400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfd221-04c2-4925-882a-de49f3c4ee37 None admin] PATH_INFO: `/v3/users/ab407cecb2b248d483a7a7fac41d4d1b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.837005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfd221-04c2-4925-882a-de49f3c4ee37 None admin] RBAC: Authorizing `identity:get_user(user_id=ab407cecb2b248d483a7a7fac41d4d1b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:48.838439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfd221-04c2-4925-882a-de49f3c4ee37 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:48.839971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 426/869] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:21:48 2022] GET /identity/v3/users/ab407cecb2b248d483a7a7fac41d4d1b => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:48.845448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:48.942247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225a2b2-8afd-4556-9221-96a4ed79b894 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:48.943666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:48.976434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01de5d4-cb11-443e-b3e9-863a3c185c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:48.977040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01de5d4-cb11-443e-b3e9-863a3c185c82 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.977320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01de5d4-cb11-443e-b3e9-863a3c185c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.977591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01de5d4-cb11-443e-b3e9-863a3c185c82 None admin] PATH_INFO: `/v3/projects/cb07b4ba7ae4468499556217362ef583` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.978713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01de5d4-cb11-443e-b3e9-863a3c185c82 None admin] RBAC: Authorizing `identity:get_project(project_id=cb07b4ba7ae4468499556217362ef583)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:48.980109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01de5d4-cb11-443e-b3e9-863a3c185c82 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:48.981693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 444/870] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:21:48 2022] GET /identity/v3/projects/cb07b4ba7ae4468499556217362ef583 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:48.987362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfd221-04c2-4925-882a-de49f3c4ee37 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:49.038941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:49.086729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfd221-04c2-4925-882a-de49f3c4ee37 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:49.118943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081fde7-ec1c-4036-b756-73c58e5f2303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:49.119583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081fde7-ec1c-4036-b756-73c58e5f2303 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:49.119858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081fde7-ec1c-4036-b756-73c58e5f2303 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:49.120128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081fde7-ec1c-4036-b756-73c58e5f2303 None admin] PATH_INFO: `/v3/projects/cb07b4ba7ae4468499556217362ef583/users/ab407cecb2b248d483a7a7fac41d4d1b/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:49.122068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081fde7-ec1c-4036-b756-73c58e5f2303 None admin] RBAC: Authorizing `identity:create_grant(project_id=cb07b4ba7ae4468499556217362ef583, user_id=ab407cecb2b248d483a7a7fac41d4d1b, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:49.123472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081fde7-ec1c-4036-b756-73c58e5f2303 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:49.208593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 427/871] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:21:48 2022] PUT /identity/v3/projects/cb07b4ba7ae4468499556217362ef583/users/ab407cecb2b248d483a7a7fac41d4d1b/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:21:51.294393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2740296-f1a0-41f5-9d0a-c2ddbb57730f None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.294916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2740296-f1a0-41f5-9d0a-c2ddbb57730f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.294916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2740296-f1a0-41f5-9d0a-c2ddbb57730f None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.296270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 445/872] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:51.301470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42346dd0-4e75-4b65-b14a-4af37530a4dd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.301867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42346dd0-4e75-4b65-b14a-4af37530a4dd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.302047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42346dd0-4e75-4b65-b14a-4af37530a4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.320034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-42346dd0-4e75-4b65-b14a-4af37530a4dd None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:51.321140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-42346dd0-4e75-4b65-b14a-4af37530a4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:51.386580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 428/873] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:51.654076 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2740296-f1a0-41f5-9d0a-c2ddbb57730f None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:51.754347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2740296-f1a0-41f5-9d0a-c2ddbb57730f None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:51.787024 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48860f1-2b1f-49c5-a855-8dffaabee6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:51.787629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48860f1-2b1f-49c5-a855-8dffaabee6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.787909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48860f1-2b1f-49c5-a855-8dffaabee6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.788180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48860f1-2b1f-49c5-a855-8dffaabee6a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.798435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48860f1-2b1f-49c5-a855-8dffaabee6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-c48860f1-2b1f-49c5-a855-8dffaabee6a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:51.803132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:51.804195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:21:51.805214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 446/874] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:51.810801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42346dd0-4e75-4b65-b14a-4af37530a4dd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:51.849918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:51.905509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42346dd0-4e75-4b65-b14a-4af37530a4dd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:51.937986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:51.938572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.938845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.939113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.939760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:51.940057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:51.941400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:51.949022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 429/875] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:51.954926 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48860f1-2b1f-49c5-a855-8dffaabee6a4 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:51.985966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:52.053115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48860f1-2b1f-49c5-a855-8dffaabee6a4 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:52.086677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10e1fe-9730-4ac6-b452-980b5c366588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:52.087288 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10e1fe-9730-4ac6-b452-980b5c366588 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:52.087566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10e1fe-9730-4ac6-b452-980b5c366588 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:52.088351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10e1fe-9730-4ac6-b452-980b5c366588 None admin] PATH_INFO: `/v3/users/ab407cecb2b248d483a7a7fac41d4d1b` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:52.089450 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10e1fe-9730-4ac6-b452-980b5c366588 None admin] RBAC: Authorizing `identity:get_user(user_id=ab407cecb2b248d483a7a7fac41d4d1b)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:52.090892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10e1fe-9730-4ac6-b452-980b5c366588 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:52.092460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 447/876] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:21:51 2022] GET /identity/v3/users/ab407cecb2b248d483a7a7fac41d4d1b => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:52.098189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:52.193442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff59b-b841-42e1-b632-36bfbe3725a2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:52.194878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:52.226790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396247a-38a8-4c94-9cf4-6cb2a1e994f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:52.227419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396247a-38a8-4c94-9cf4-6cb2a1e994f7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:52.227967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396247a-38a8-4c94-9cf4-6cb2a1e994f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:52.228234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396247a-38a8-4c94-9cf4-6cb2a1e994f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:52.291551 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396247a-38a8-4c94-9cf4-6cb2a1e994f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:52.292971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396247a-38a8-4c94-9cf4-6cb2a1e994f7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:52.385707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 430/877] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:21:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:21:52.391389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10e1fe-9730-4ac6-b452-980b5c366588 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:52.484733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:52.488315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10e1fe-9730-4ac6-b452-980b5c366588 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:52.520709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95a03e-3ffc-491a-97d7-576912608080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:52.521312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95a03e-3ffc-491a-97d7-576912608080 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:52.521589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95a03e-3ffc-491a-97d7-576912608080 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:52.521859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95a03e-3ffc-491a-97d7-576912608080 None admin] PATH_INFO: `/v3/projects/cb07b4ba7ae4468499556217362ef583` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:52.522981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95a03e-3ffc-491a-97d7-576912608080 None admin] RBAC: Authorizing `identity:get_project(project_id=cb07b4ba7ae4468499556217362ef583)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:52.524725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95a03e-3ffc-491a-97d7-576912608080 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:52.526742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 448/878] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:21:52 2022] GET /identity/v3/projects/cb07b4ba7ae4468499556217362ef583 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:52.532424 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396247a-38a8-4c94-9cf4-6cb2a1e994f7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:52.534546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:52.628599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396247a-38a8-4c94-9cf4-6cb2a1e994f7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:52.661022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db621c39-670d-45bc-b232-85fef8a63189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:52.661601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db621c39-670d-45bc-b232-85fef8a63189 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:52.661875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db621c39-670d-45bc-b232-85fef8a63189 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:52.662169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db621c39-670d-45bc-b232-85fef8a63189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:52.662876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db621c39-670d-45bc-b232-85fef8a63189 None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=cb07b4ba7ae4468499556217362ef583, user.id=ab407cecb2b248d483a7a7fac41d4d1b) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:52.663177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db621c39-670d-45bc-b232-85fef8a63189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:52.664560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db621c39-670d-45bc-b232-85fef8a63189 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:52.670689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 431/879] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:21:52 2022] GET /identity/v3/role_assignments?user.id=ab407cecb2b248d483a7a7fac41d4d1b&scope.project.id=cb07b4ba7ae4468499556217362ef583&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:54.777111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aee94-98e0-43b4-a8b8-e784dc6505ef None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:54.777641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aee94-98e0-43b4-a8b8-e784dc6505ef None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:54.777641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aee94-98e0-43b4-a8b8-e784dc6505ef None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:54.779047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 449/880] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:54.784258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3038d4-cc0e-4355-9870-7398ec01094b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:54.784696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3038d4-cc0e-4355-9870-7398ec01094b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:54.784787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3038d4-cc0e-4355-9870-7398ec01094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:54.802504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-db3038d4-cc0e-4355-9870-7398ec01094b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:54.803668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-db3038d4-cc0e-4355-9870-7398ec01094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:54.860460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:54.867568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 432/881] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:55.137048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aee94-98e0-43b4-a8b8-e784dc6505ef None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:55.234095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aee94-98e0-43b4-a8b8-e784dc6505ef None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:55.266297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06607cd-578f-4376-ab1f-ab10d39171ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:55.266878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06607cd-578f-4376-ab1f-ab10d39171ea None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:55.267152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06607cd-578f-4376-ab1f-ab10d39171ea None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:55.267420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06607cd-578f-4376-ab1f-ab10d39171ea None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:55.268073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06607cd-578f-4376-ab1f-ab10d39171ea None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:55.269404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06607cd-578f-4376-ab1f-ab10d39171ea None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:55.367740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 450/882] 10.222.0.44 () {60 vars in 1138 bytes} [Wed Oct 12 12:21:55 2022] POST /identity/v3/domains => generated 267 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:57.489966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a840970-076d-4e7b-addb-dfcf88befa40 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:57.490481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a840970-076d-4e7b-addb-dfcf88befa40 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:57.490481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a840970-076d-4e7b-addb-dfcf88befa40 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:57.491765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 433/883] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:57.497620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2310447-04b1-45b0-8f91-354f1ae0e93d None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:57.498006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2310447-04b1-45b0-8f91-354f1ae0e93d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:57.498202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2310447-04b1-45b0-8f91-354f1ae0e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:57.515780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e2310447-04b1-45b0-8f91-354f1ae0e93d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:57.516944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e2310447-04b1-45b0-8f91-354f1ae0e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:57.536157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:57.580963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 451/884] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:57.848934 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a840970-076d-4e7b-addb-dfcf88befa40 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:57.946185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a840970-076d-4e7b-addb-dfcf88befa40 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:57.979128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c2ef0-dc0a-4412-9829-34d6e203a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:57.979727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c2ef0-dc0a-4412-9829-34d6e203a40b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:57.979999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c2ef0-dc0a-4412-9829-34d6e203a40b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:57.980259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c2ef0-dc0a-4412-9829-34d6e203a40b None admin] PATH_INFO: `/v3/domains/e20bd063ccb14616aa6774ded789a4e6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:57.990269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c2ef0-dc0a-4412-9829-34d6e203a40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=e20bd063ccb14616aa6774ded789a4e6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:57.991673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c2ef0-dc0a-4412-9829-34d6e203a40b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:57.993328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 434/885] 10.222.0.44 () {56 vars in 1183 bytes} [Wed Oct 12 12:21:57 2022] GET /identity/v3/domains/e20bd063ccb14616aa6774ded789a4e6 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:57.999175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2310447-04b1-45b0-8f91-354f1ae0e93d None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:58.094163 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2310447-04b1-45b0-8f91-354f1ae0e93d None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:58.127406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a59631-03af-4560-961f-7de5b8f452a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:58.127946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a59631-03af-4560-961f-7de5b8f452a2 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:58.128221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a59631-03af-4560-961f-7de5b8f452a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:58.128486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a59631-03af-4560-961f-7de5b8f452a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:58.129113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a59631-03af-4560-961f-7de5b8f452a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:58.130549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a59631-03af-4560-961f-7de5b8f452a2 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:58.247240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 452/886] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:57 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:22:00.384694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9538ea-2d7c-4a8e-a1a4-32f602ae1df3 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:00.385233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9538ea-2d7c-4a8e-a1a4-32f602ae1df3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:00.385233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9538ea-2d7c-4a8e-a1a4-32f602ae1df3 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:00.386514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 435/887] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:22:00.391749 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e6cec-038e-49a6-a8a1-44f6cb43a9a3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:00.392192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e6cec-038e-49a6-a8a1-44f6cb43a9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:00.392308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e6cec-038e-49a6-a8a1-44f6cb43a9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:00.410720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c11e6cec-038e-49a6-a8a1-44f6cb43a9a3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:22:00.411933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c11e6cec-038e-49a6-a8a1-44f6cb43a9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:00.413825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:00.478865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 453/888] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:22:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:22:00.743660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9538ea-2d7c-4a8e-a1a4-32f602ae1df3 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:00.798317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:00.840544 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9538ea-2d7c-4a8e-a1a4-32f602ae1df3 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:00.872812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b00d1-90da-4b85-a79b-99ece136fb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:00.873413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b00d1-90da-4b85-a79b-99ece136fb25 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:00.873687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b00d1-90da-4b85-a79b-99ece136fb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:00.873974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b00d1-90da-4b85-a79b-99ece136fb25 None admin] PATH_INFO: `/v3/domains/e20bd063ccb14616aa6774ded789a4e6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:00.875083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b00d1-90da-4b85-a79b-99ece136fb25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e20bd063ccb14616aa6774ded789a4e6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:00.876471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b00d1-90da-4b85-a79b-99ece136fb25 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:00.878005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 436/889] 10.222.0.44 () {56 vars in 1183 bytes} [Wed Oct 12 12:22:00 2022] GET /identity/v3/domains/e20bd063ccb14616aa6774ded789a4e6 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:22:00.883603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e6cec-038e-49a6-a8a1-44f6cb43a9a3 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:00.978229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e6cec-038e-49a6-a8a1-44f6cb43a9a3 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:01.011429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5b537-5c31-48de-b7df-b4b6c40cc4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:01.011959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5b537-5c31-48de-b7df-b4b6c40cc4ed None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:01.012234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5b537-5c31-48de-b7df-b4b6c40cc4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:01.012502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5b537-5c31-48de-b7df-b4b6c40cc4ed None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:01.013145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5b537-5c31-48de-b7df-b4b6c40cc4ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:01.014568 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5b537-5c31-48de-b7df-b4b6c40cc4ed None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:01.024114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:01.109507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 454/890] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:22:00 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:22:03.235145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb416bbd-1e54-4953-8123-2de7b6ffc876 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:03.235704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb416bbd-1e54-4953-8123-2de7b6ffc876 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:03.235704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb416bbd-1e54-4953-8123-2de7b6ffc876 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:03.237030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 437/891] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:22:03.242192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018baa-0fda-4832-8921-fd98d1bd2540 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:03.242636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018baa-0fda-4832-8921-fd98d1bd2540 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:03.242736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018baa-0fda-4832-8921-fd98d1bd2540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:03.260955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-80018baa-0fda-4832-8921-fd98d1bd2540 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:22:03.262155 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-80018baa-0fda-4832-8921-fd98d1bd2540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:03.325311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 455/892] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:22:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:22:03.590117 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb416bbd-1e54-4953-8123-2de7b6ffc876 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:03.687575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb416bbd-1e54-4953-8123-2de7b6ffc876 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:03.709593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:03.719628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b0c6d-1961-456d-b8b4-870053ca3b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:03.722055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b0c6d-1961-456d-b8b4-870053ca3b0e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:03.722055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b0c6d-1961-456d-b8b4-870053ca3b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:03.722055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b0c6d-1961-456d-b8b4-870053ca3b0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:03.730994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b0c6d-1961-456d-b8b4-870053ca3b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-9c7b0c6d-1961-456d-b8b4-870053ca3b0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:22:03.735722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:22:03.736851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:22:03.737947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 438/893] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:22:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:22:03.742875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80018baa-0fda-4832-8921-fd98d1bd2540 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:03.839045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80018baa-0fda-4832-8921-fd98d1bd2540 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:03.862114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:03.872867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:03.876474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:03.876474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:03.876474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:03.876474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:22:03.876474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:03.876474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:03.884034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 456/894] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:22:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:22:03.889666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b0c6d-1961-456d-b8b4-870053ca3b0e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:03.985097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b0c6d-1961-456d-b8b4-870053ca3b0e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:04.016851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b90c8-1f7b-40a9-ae1c-c6a46233cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:04.017455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b90c8-1f7b-40a9-ae1c-c6a46233cfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:04.017731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b90c8-1f7b-40a9-ae1c-c6a46233cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:04.018029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b90c8-1f7b-40a9-ae1c-c6a46233cfe2 None admin] PATH_INFO: `/v3/users/802557745b684bc399132a5765c4fb09` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:04.030582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b90c8-1f7b-40a9-ae1c-c6a46233cfe2 None admin] RBAC: Authorizing `identity:get_user(user_id=802557745b684bc399132a5765c4fb09)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:04.032000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b90c8-1f7b-40a9-ae1c-c6a46233cfe2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:04.033545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 439/895] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:22:03 2022] GET /identity/v3/users/802557745b684bc399132a5765c4fb09 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:22:04.039038 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:04.137597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7ddfb-de19-4aa8-8c1a-23d7f1fe8cc5 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:04.170018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb823d5e-2a4f-441a-86b5-497d51cd07c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:04.170638 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb823d5e-2a4f-441a-86b5-497d51cd07c0 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:04.170917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb823d5e-2a4f-441a-86b5-497d51cd07c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:04.171186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb823d5e-2a4f-441a-86b5-497d51cd07c0 None admin] PATH_INFO: `/v3/projects/fdf8753a3a52488cb128b0f60ce7645a` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:04.172284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb823d5e-2a4f-441a-86b5-497d51cd07c0 None admin] RBAC: Authorizing `identity:get_project(project_id=fdf8753a3a52488cb128b0f60ce7645a)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:04.173690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb823d5e-2a4f-441a-86b5-497d51cd07c0 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:04.175323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 457/896] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:22:04 2022] GET /identity/v3/projects/fdf8753a3a52488cb128b0f60ce7645a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:22:04.181155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b90c8-1f7b-40a9-ae1c-c6a46233cfe2 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:04.222570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:04.277334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b90c8-1f7b-40a9-ae1c-c6a46233cfe2 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:04.309866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a62e2-8e68-42c2-95d0-e77fab33b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:04.310531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a62e2-8e68-42c2-95d0-e77fab33b5cd None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:04.310810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a62e2-8e68-42c2-95d0-e77fab33b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:04.311082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a62e2-8e68-42c2-95d0-e77fab33b5cd None admin] PATH_INFO: `/v3/projects/fdf8753a3a52488cb128b0f60ce7645a/users/802557745b684bc399132a5765c4fb09/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:04.312936 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a62e2-8e68-42c2-95d0-e77fab33b5cd None admin] RBAC: Authorizing `identity:create_grant(project_id=fdf8753a3a52488cb128b0f60ce7645a, user_id=802557745b684bc399132a5765c4fb09, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:04.314381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a62e2-8e68-42c2-95d0-e77fab33b5cd None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:04.383299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 440/897] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:22:04 2022] PUT /identity/v3/projects/fdf8753a3a52488cb128b0f60ce7645a/users/802557745b684bc399132a5765c4fb09/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:22:06.466585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653428d8-a75b-4434-bb8b-be683f763950 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:06.467101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653428d8-a75b-4434-bb8b-be683f763950 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:06.467101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653428d8-a75b-4434-bb8b-be683f763950 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:06.468405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 458/898] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:22:06.473766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e421888-bf1f-488b-9c52-fc1651180243 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:06.474199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e421888-bf1f-488b-9c52-fc1651180243 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:06.474596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e421888-bf1f-488b-9c52-fc1651180243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:06.493613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6e421888-bf1f-488b-9c52-fc1651180243 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:22:06.494836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6e421888-bf1f-488b-9c52-fc1651180243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:06.558241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 441/899] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:22:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:22:06.822793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653428d8-a75b-4434-bb8b-be683f763950 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:06.878285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:06.923342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653428d8-a75b-4434-bb8b-be683f763950 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:06.956129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280950ec-c6a9-4919-ac97-9ae4968d024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:06.956746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280950ec-c6a9-4919-ac97-9ae4968d024b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:06.957026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280950ec-c6a9-4919-ac97-9ae4968d024b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:06.957302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280950ec-c6a9-4919-ac97-9ae4968d024b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:06.967084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280950ec-c6a9-4919-ac97-9ae4968d024b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-280950ec-c6a9-4919-ac97-9ae4968d024b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:22:06.972296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:22:06.973361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:22:06.974262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 459/900] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:22:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:22:06.979722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e421888-bf1f-488b-9c52-fc1651180243 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:07.008844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:07.075442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e421888-bf1f-488b-9c52-fc1651180243 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:07.107744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:07.108331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:07.108605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:07.108871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:07.109517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:22:07.109815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:07.111193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:07.118757 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 442/901] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:22:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:22:07.125115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280950ec-c6a9-4919-ac97-9ae4968d024b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:07.220868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280950ec-c6a9-4919-ac97-9ae4968d024b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:07.234166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:07.253639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defc953-80cb-4c1d-a42f-f5c911d60e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:07.254276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defc953-80cb-4c1d-a42f-f5c911d60e8d None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:07.254557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defc953-80cb-4c1d-a42f-f5c911d60e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:07.254831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defc953-80cb-4c1d-a42f-f5c911d60e8d None admin] PATH_INFO: `/v3/users/802557745b684bc399132a5765c4fb09` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:07.255945 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defc953-80cb-4c1d-a42f-f5c911d60e8d None admin] RBAC: Authorizing `identity:get_user(user_id=802557745b684bc399132a5765c4fb09)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:07.257365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defc953-80cb-4c1d-a42f-f5c911d60e8d None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:07.258955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 460/902] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Oct 12 12:22:07 2022] GET /identity/v3/users/802557745b684bc399132a5765c4fb09 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:22:07.265144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:07.352904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:07.359599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58febb4c-9169-4694-bb08-63682ac39f16 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:07.392049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4a151-7d3c-4528-bee1-018b48d0c6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:07.392638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4a151-7d3c-4528-bee1-018b48d0c6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:07.392916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4a151-7d3c-4528-bee1-018b48d0c6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:07.393187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4a151-7d3c-4528-bee1-018b48d0c6e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:07.456856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4a151-7d3c-4528-bee1-018b48d0c6e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:07.458307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4a151-7d3c-4528-bee1-018b48d0c6e4 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:07.544758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:07.551534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 443/903] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:22:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:22:07.557953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defc953-80cb-4c1d-a42f-f5c911d60e8d None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:07.654573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defc953-80cb-4c1d-a42f-f5c911d60e8d None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:07.686942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359c673-a807-4a1c-a47f-70ba94bc2fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:07.687552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359c673-a807-4a1c-a47f-70ba94bc2fda None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:07.687830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359c673-a807-4a1c-a47f-70ba94bc2fda None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:07.688101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359c673-a807-4a1c-a47f-70ba94bc2fda None admin] PATH_INFO: `/v3/projects/fdf8753a3a52488cb128b0f60ce7645a` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:07.689191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359c673-a807-4a1c-a47f-70ba94bc2fda None admin] RBAC: Authorizing `identity:get_project(project_id=fdf8753a3a52488cb128b0f60ce7645a)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:07.690632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359c673-a807-4a1c-a47f-70ba94bc2fda None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:07.692225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 461/904] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Oct 12 12:22:07 2022] GET /identity/v3/projects/fdf8753a3a52488cb128b0f60ce7645a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:22:07.698240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4a151-7d3c-4528-bee1-018b48d0c6e4 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:07.792099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4a151-7d3c-4528-bee1-018b48d0c6e4 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:07.823554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d978-c206-448b-b534-a670753d5b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:07.824134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d978-c206-448b-b534-a670753d5b59 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:07.824407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d978-c206-448b-b534-a670753d5b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:07.824673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d978-c206-448b-b534-a670753d5b59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:07.825379 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d978-c206-448b-b534-a670753d5b59 None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=fdf8753a3a52488cb128b0f60ce7645a, user.id=802557745b684bc399132a5765c4fb09) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:22:07.825682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d978-c206-448b-b534-a670753d5b59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:07.827095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d978-c206-448b-b534-a670753d5b59 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:07.832589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 444/905] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:22:07 2022] GET /identity/v3/role_assignments?user.id=802557745b684bc399132a5765c4fb09&scope.project.id=fdf8753a3a52488cb128b0f60ce7645a&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:22:17.682959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=75 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:17.830152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=71 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:19.990390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473fb81-c467-4073-98da-0d189157a85d None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:19.991034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473fb81-c467-4073-98da-0d189157a85d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:19.991034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473fb81-c467-4073-98da-0d189157a85d None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:19.992303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 462/906] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:19.997410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863b661-3d7e-4c0a-96d3-215bb4fb1be0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:19.997843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863b661-3d7e-4c0a-96d3-215bb4fb1be0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:19.998069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863b661-3d7e-4c0a-96d3-215bb4fb1be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:20.016537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5863b661-3d7e-4c0a-96d3-215bb4fb1be0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:20.017707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5863b661-3d7e-4c0a-96d3-215bb4fb1be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:20.087115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 445/907] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:20.358895 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473fb81-c467-4073-98da-0d189157a85d None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:20.459026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473fb81-c467-4073-98da-0d189157a85d None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:20.492626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:20.493241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:20.493518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:20.493790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:20.503579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:20.504996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:20.519340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:20.527011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:20.528169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:20.528169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:20.528169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:20.528169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 463/908] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:20.528460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863b661-3d7e-4c0a-96d3-215bb4fb1be0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:20.623333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863b661-3d7e-4c0a-96d3-215bb4fb1be0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:20.651378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:20.657008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e2076-0db4-48de-ae8b-d14d6c315ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:20.657608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e2076-0db4-48de-ae8b-d14d6c315ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:20.657885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e2076-0db4-48de-ae8b-d14d6c315ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:20.658186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e2076-0db4-48de-ae8b-d14d6c315ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:20.658820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e2076-0db4-48de-ae8b-d14d6c315ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:20.659120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e2076-0db4-48de-ae8b-d14d6c315ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:20.660464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e2076-0db4-48de-ae8b-d14d6c315ba5 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:20.672188 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 446/909] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:20.677868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:20.729766 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:20.774637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf127bda-1ffd-4c1c-af5a-8c4523b99de5 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:20.806888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d593a-17b4-41b8-b597-2e8e0de161ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:20.807486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d593a-17b4-41b8-b597-2e8e0de161ec None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:20.807765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d593a-17b4-41b8-b597-2e8e0de161ec None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:20.808032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d593a-17b4-41b8-b597-2e8e0de161ec None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:20.808663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d593a-17b4-41b8-b597-2e8e0de161ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:20.810099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d593a-17b4-41b8-b597-2e8e0de161ec None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:20.913281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 464/910] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Oct 12 12:23:20 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:23:23.081701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411384a4-d006-4bed-8866-5978217baa77 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:23.085481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411384a4-d006-4bed-8866-5978217baa77 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:23.085481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411384a4-d006-4bed-8866-5978217baa77 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:23.085481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 447/911] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:23.089363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d32e1-0700-4fd7-95fe-324a12087715 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:23.089754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d32e1-0700-4fd7-95fe-324a12087715 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:23.089911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d32e1-0700-4fd7-95fe-324a12087715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:23.109417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9e5d32e1-0700-4fd7-95fe-324a12087715 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:23.114196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9e5d32e1-0700-4fd7-95fe-324a12087715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:23.176081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 465/912] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:23.454373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411384a4-d006-4bed-8866-5978217baa77 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:23.553705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411384a4-d006-4bed-8866-5978217baa77 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:23.586422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39bc08-c4cc-4b4f-b0bb-90c27464c2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:23.587119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39bc08-c4cc-4b4f-b0bb-90c27464c2dc None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:23.587393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39bc08-c4cc-4b4f-b0bb-90c27464c2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:23.587661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39bc08-c4cc-4b4f-b0bb-90c27464c2dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:23.597880 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39bc08-c4cc-4b4f-b0bb-90c27464c2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-2d39bc08-c4cc-4b4f-b0bb-90c27464c2dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:23:23.603052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:23.604303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:23.605316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 448/913] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:23:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:23.610263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d32e1-0700-4fd7-95fe-324a12087715 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:23.649221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:23.707144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d32e1-0700-4fd7-95fe-324a12087715 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:23.741193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:23.741791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:23.742095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:23.742364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:23.743015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] RBAC: Adding query filter params (name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:23.743318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:23.744664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:23.752533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 466/914] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:23:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:23.759064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39bc08-c4cc-4b4f-b0bb-90c27464c2dc None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:23.807164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:23.855297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39bc08-c4cc-4b4f-b0bb-90c27464c2dc None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:23.888287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:23.888906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:23.889186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:23.889458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:23.899943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:23.901403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:23.915471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:23.916591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:23.917875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:23.917875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:23.917875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:23.917875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 449/915] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:23.922492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:24.020901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:24.023892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b819ae7-0107-4540-a453-a3d6b5db8f2a None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:24.056811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:24.057400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:24.057674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:24.057951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:24.058587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:24.058885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:24.060217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:24.071573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 467/916] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:24.077301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:24.174574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc0542-d6cf-4d8c-9614-754d5ba88bc7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:24.176032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:24.206763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:24.207373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:24.207648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:24.207917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:24.215039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:24.216454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:23:24.227968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:24.229067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:24.230490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 450/917] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:24.234860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:24.332702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2d354-92bc-4104-972c-71f398ba1682 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:24.366837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:24.367432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:24.367710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:24.367980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:24.368698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:24.369002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:24.370452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:24.386915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 468/918] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Oct 12 12:23:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:24.392943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:24.492200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c07443-31f3-4bc0-b9f1-8d3779a4efd9 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:24.525055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:24.525675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:24.525974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:24.526251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:24.536417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:24.541124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:24.552053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:24.553061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:24.553946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:24.553946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:24.553946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:24.553946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 451/919] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:24.559114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:24.577920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:24.658782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce8774-775f-4282-9581-f93ed5a774b1 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:24.691615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:24.692211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:24.692490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:24.693251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:24.693885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:24.697575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:24.697575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:24.706629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 469/920] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:24.712871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:24.728878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:24.810206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8f947-19ca-4cbe-a8bc-c46665319543 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:24.842873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:24.843491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:24.843772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:24.844040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:24.850398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:24.852083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:23:24.863053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:24.864020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:24.865147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 452/921] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:24.870103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:24.947819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:24.969093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d9470-592a-4974-9983-e7128390e13f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:25.001521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c58bc-45d0-4341-8810-1c705c1bd86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:25.002141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c58bc-45d0-4341-8810-1c705c1bd86b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:25.002419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c58bc-45d0-4341-8810-1c705c1bd86b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:25.002686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c58bc-45d0-4341-8810-1c705c1bd86b None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:25.003362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c58bc-45d0-4341-8810-1c705c1bd86b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:25.003663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c58bc-45d0-4341-8810-1c705c1bd86b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:25.005114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c58bc-45d0-4341-8810-1c705c1bd86b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:25.017323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 470/922] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:23:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:25.023630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:25.099464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:25.123524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894df151-1a0b-4c82-b8f3-03cf9a2f7247 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:25.155280 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438133f-2b79-4623-ab20-079713f25ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:25.155917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438133f-2b79-4623-ab20-079713f25ba8 None admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:25.156190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438133f-2b79-4623-ab20-079713f25ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:25.156458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438133f-2b79-4623-ab20-079713f25ba8 None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/f35e7add10924dc49eb9d4a983c89eb3/roles/7fcd00bd7c614a9587d3161df27b1cff` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:25.170465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438133f-2b79-4623-ab20-079713f25ba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=f35e7add10924dc49eb9d4a983c89eb3, role_id=7fcd00bd7c614a9587d3161df27b1cff)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:25.171878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438133f-2b79-4623-ab20-079713f25ba8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:25.265888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 453/923] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:23:25 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/f35e7add10924dc49eb9d4a983c89eb3/roles/7fcd00bd7c614a9587d3161df27b1cff => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:23:27.381197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06277fc3-761f-48c5-88e9-3247ffa7382c None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:27.381712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06277fc3-761f-48c5-88e9-3247ffa7382c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:27.381712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06277fc3-761f-48c5-88e9-3247ffa7382c None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:27.383027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 471/924] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:27.388185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f2c39-0997-4ea4-8259-99b6c80f239b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:27.388629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f2c39-0997-4ea4-8259-99b6c80f239b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:27.388722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f2c39-0997-4ea4-8259-99b6c80f239b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:27.406963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-571f2c39-0997-4ea4-8259-99b6c80f239b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:27.408135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-571f2c39-0997-4ea4-8259-99b6c80f239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:27.472030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 454/925] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:27.737153 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06277fc3-761f-48c5-88e9-3247ffa7382c None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:27.841148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06277fc3-761f-48c5-88e9-3247ffa7382c None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:27.873980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8a712-cf49-4533-b4b8-25a3c600f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:27.874671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8a712-cf49-4533-b4b8-25a3c600f014 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:27.874946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8a712-cf49-4533-b4b8-25a3c600f014 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:27.875212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8a712-cf49-4533-b4b8-25a3c600f014 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:27.882231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:27.885964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8a712-cf49-4533-b4b8-25a3c600f014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-98f8a712-cf49-4533-b4b8-25a3c600f014 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:23:27.890706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:27.891782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:27.892609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 472/926] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Oct 12 12:23:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:27.898537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f2c39-0997-4ea4-8259-99b6c80f239b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:27.950818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:27.992685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f2c39-0997-4ea4-8259-99b6c80f239b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:28.025410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:28.026029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:28.026307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:28.026575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:28.027483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:28.027765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:28.029431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:28.037186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 455/927] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Oct 12 12:23:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:28.043103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8a712-cf49-4533-b4b8-25a3c600f014 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:28.144959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8a712-cf49-4533-b4b8-25a3c600f014 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:28.179004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:28.179625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:28.179907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:28.180181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:28.190447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:28.191862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:28.206378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:28.207469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:28.208751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:28.208751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:28.208751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:28.208751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 473/928] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:28.213167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:28.310219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bdf15-75e7-43b7-b58c-15272500eb10 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:28.326341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:28.342986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:28.343585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:28.343864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:28.344133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:28.344765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:28.345081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:28.346455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:28.357870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 456/929] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:28.363718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:28.414198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:28.461840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290934-d17e-4886-86a0-c5947024ad48 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:28.495819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:28.496456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:28.496732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:28.496995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:28.504149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:28.505595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:23:28.517844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:28.519098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:28.520412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 474/930] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:28.524869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:28.623983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3603a4f-c332-4904-bb4a-dbc9a7ae3ea6 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:28.657073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:28.657670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:28.657972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:28.658249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:28.658971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:28.659279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:28.660760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:28.679007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 457/931] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Oct 12 12:23:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:28.685014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:28.784705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c965d-57ab-4bc4-9aaf-76a8532dc72b None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:28.795594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:28.817889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9a7d4-73bb-4d4b-b9d3-15101b4363f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:28.818509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a7d4-73bb-4d4b-b9d3-15101b4363f5 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:28.818804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a7d4-73bb-4d4b-b9d3-15101b4363f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:28.819055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a7d4-73bb-4d4b-b9d3-15101b4363f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:28.885285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9a7d4-73bb-4d4b-b9d3-15101b4363f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:28.886674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9a7d4-73bb-4d4b-b9d3-15101b4363f5 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:28.983343 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 475/932] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:23:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:23:28.989164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:29.024058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:29.088764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8457789-ad2f-4519-b99a-6972e181f1ce None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:29.122450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:29.123069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:29.123343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:29.123617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:29.133723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:29.135191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:29.149785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:29.150964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:29.151866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:29.151866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:29.151866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:29.151866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 458/933] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:29.156604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9a7d4-73bb-4d4b-b9d3-15101b4363f5 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:29.167959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:29.254073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9a7d4-73bb-4d4b-b9d3-15101b4363f5 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:29.287855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:29.288454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:29.288733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:29.289004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:29.289636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:29.289971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:29.291323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:29.302415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 476/934] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:29.308132 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:29.392345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:29.407212 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b3a51-293d-464e-9be7-0fba8dd8f9c1 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:29.439812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:29.440425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:29.440703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:29.440973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:29.447915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:29.452442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:23:29.460865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:29.461971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:29.463054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 459/935] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:23:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:29.467865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:29.537801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:29.568694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bdb84-f73a-4042-8d37-5cd15ec306b4 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:29.600945 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1b9e0-0eac-4c6d-ab70-caf02277a1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:29.601788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b9e0-0eac-4c6d-ab70-caf02277a1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:29.602095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b9e0-0eac-4c6d-ab70-caf02277a1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:29.602464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1b9e0-0eac-4c6d-ab70-caf02277a1f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:29.603149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1b9e0-0eac-4c6d-ab70-caf02277a1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:29.603451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1b9e0-0eac-4c6d-ab70-caf02277a1f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:29.604890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1b9e0-0eac-4c6d-ab70-caf02277a1f4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:29.617237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 477/936] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:23:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:23:29.623230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:29.718826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f1a2-4dc3-48a4-9058-38fcdca73ec7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:29.751044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae88c4-f1cc-42c6-8498-1882d6aee10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:29.751615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae88c4-f1cc-42c6-8498-1882d6aee10d None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:29.751889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae88c4-f1cc-42c6-8498-1882d6aee10d None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:29.752154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae88c4-f1cc-42c6-8498-1882d6aee10d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:29.752868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae88c4-f1cc-42c6-8498-1882d6aee10d None admin] RBAC: Adding query filter params (role.id=7fcd00bd7c614a9587d3161df27b1cff, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=f35e7add10924dc49eb9d4a983c89eb3) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:29.753169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae88c4-f1cc-42c6-8498-1882d6aee10d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:29.754602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae88c4-f1cc-42c6-8498-1882d6aee10d None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:29.760602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 460/937] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:23:29 2022] GET /identity/v3/role_assignments?user.id=f35e7add10924dc49eb9d4a983c89eb3&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7fcd00bd7c614a9587d3161df27b1cff => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:31.908312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd225e8d-6394-4ab8-a637-b6c05d20efdd None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:31.908862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd225e8d-6394-4ab8-a637-b6c05d20efdd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:31.908862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd225e8d-6394-4ab8-a637-b6c05d20efdd None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:31.910250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 478/938] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:31.916330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bac3be-c311-43b7-8f2d-3bb612a473a6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:31.916736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bac3be-c311-43b7-8f2d-3bb612a473a6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:31.916870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bac3be-c311-43b7-8f2d-3bb612a473a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:31.934817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-60bac3be-c311-43b7-8f2d-3bb612a473a6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:31.936011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-60bac3be-c311-43b7-8f2d-3bb612a473a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:31.938812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:32.002377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 461/939] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:32.269871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd225e8d-6394-4ab8-a637-b6c05d20efdd None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:32.369404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd225e8d-6394-4ab8-a637-b6c05d20efdd None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:32.398244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:32.403919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de99649-15a8-4f75-abe4-c4191beec709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:32.404536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99649-15a8-4f75-abe4-c4191beec709 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:32.404829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99649-15a8-4f75-abe4-c4191beec709 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:32.405084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99649-15a8-4f75-abe4-c4191beec709 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:32.415126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de99649-15a8-4f75-abe4-c4191beec709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-2de99649-15a8-4f75-abe4-c4191beec709 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:23:32.419903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:32.421054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:32.422217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 479/940] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:32.426966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bac3be-c311-43b7-8f2d-3bb612a473a6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:32.522545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bac3be-c311-43b7-8f2d-3bb612a473a6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:32.556218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:32.556815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:32.557090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:32.557356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:32.558031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:32.558330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:32.559720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:32.567546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 462/941] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:32.573991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de99649-15a8-4f75-abe4-c4191beec709 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:32.670886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de99649-15a8-4f75-abe4-c4191beec709 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:32.704711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:32.705323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:32.705602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:32.705873 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:32.715682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:32.717099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:32.732315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:32.733417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:32.734778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:32.734778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:32.734778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:32.734778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 480/942] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:32.739223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:32.742705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:32.835498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084c58d-c64b-49f3-86f9-6b3ea060d96b None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:32.869046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:32.869638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:32.869915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:32.870218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:32.870848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:32.871148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:32.872494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:32.884056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 463/943] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:32.889731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:32.932761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:32.986500 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dd532-c38e-444f-9f1d-ff3875fde79f None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:33.019505 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:33.020123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.020400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.020671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:33.032220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:33.033639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:23:33.045023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:33.046197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:33.047357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 481/944] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:33.051957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:33.113976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:33.150754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea8b38-a946-4580-9177-240f79f430c7 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:33.184743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:33.185331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.185609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.185879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:33.186618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:33.186921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:33.188322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:33.204693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 464/945] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Oct 12 12:23:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:33.210519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:33.309178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb5c0f-8fa8-435d-9398-e22f3f5e6505 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:33.310698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:33.342488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:33.343101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.343378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.343648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:33.354683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:33.356149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:33.374587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:33.375704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:33.376791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:33.376791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:33.376791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:33.376791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 482/946] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:33.377634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:33.473097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb87d-56c2-4db9-8a1a-fdc191505fc0 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:33.492890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:33.505997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:33.506617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.506894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.507165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:33.507841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:33.508138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:33.509495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:33.520311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 465/947] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:33.526574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:33.625123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035362b-5c3c-47ae-a001-67b13461165a None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:33.658276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:33.658903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.659168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.659436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:33.665866 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:33.667318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:23:33.678263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:33.679318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:33.680263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 483/948] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:23:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:23:33.685249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:33.781752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaae6ca-22b0-4be2-b521-26c97373f905 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:33.813251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0847b-8620-4267-a91e-72919a3c64d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:33.813836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0847b-8620-4267-a91e-72919a3c64d2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.814139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0847b-8620-4267-a91e-72919a3c64d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.814408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0847b-8620-4267-a91e-72919a3c64d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:33.815079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0847b-8620-4267-a91e-72919a3c64d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:33.815541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0847b-8620-4267-a91e-72919a3c64d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:33.816946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0847b-8620-4267-a91e-72919a3c64d2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:33.828931 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 466/949] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:33.835441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:33.851517 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:33.933727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655ccb-2d7a-4da5-8bfc-0a5059596bff None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:33.968039 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d034f-4b91-440c-889c-f3563a815234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:33.968684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d034f-4b91-440c-889c-f3563a815234 None admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.968959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d034f-4b91-440c-889c-f3563a815234 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.969231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d034f-4b91-440c-889c-f3563a815234 None admin] PATH_INFO: `/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/f35e7add10924dc49eb9d4a983c89eb3/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:33.972505 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d034f-4b91-440c-889c-f3563a815234 None admin] RBAC: Authorizing `identity:create_grant(project_id=e28d11b103f94db0ae7c772a2f086978, user_id=f35e7add10924dc49eb9d4a983c89eb3, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:33.973980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d034f-4b91-440c-889c-f3563a815234 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:34.056861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 484/950] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Oct 12 12:23:33 2022] PUT /identity/v3/projects/e28d11b103f94db0ae7c772a2f086978/users/f35e7add10924dc49eb9d4a983c89eb3/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:23:36.162752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b4838-b27b-4360-a19d-9250569f976d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:36.163302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b4838-b27b-4360-a19d-9250569f976d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:36.163302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b4838-b27b-4360-a19d-9250569f976d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:36.164624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 467/951] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:36.169874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff64d6-64bb-4e0b-bc0f-4185ef9b838a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:36.170356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff64d6-64bb-4e0b-bc0f-4185ef9b838a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:36.170463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff64d6-64bb-4e0b-bc0f-4185ef9b838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:36.188089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-05ff64d6-64bb-4e0b-bc0f-4185ef9b838a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:36.189351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-05ff64d6-64bb-4e0b-bc0f-4185ef9b838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:36.255742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 485/952] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:36.524220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b4838-b27b-4360-a19d-9250569f976d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:36.565864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:36.622982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b4838-b27b-4360-a19d-9250569f976d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:36.655348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcb16b-65d4-478b-9735-7b9619883b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:36.655977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcb16b-65d4-478b-9735-7b9619883b4a None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:36.656836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcb16b-65d4-478b-9735-7b9619883b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:36.657094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dcb16b-65d4-478b-9735-7b9619883b4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:36.667631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dcb16b-65d4-478b-9735-7b9619883b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-b6dcb16b-65d4-478b-9735-7b9619883b4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:23:36.672384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:36.673474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:36.674559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 468/953] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Oct 12 12:23:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:36.680176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff64d6-64bb-4e0b-bc0f-4185ef9b838a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:36.683262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:36.776138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff64d6-64bb-4e0b-bc0f-4185ef9b838a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:36.809656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:36.810290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:36.810572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:36.810846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:36.811503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:36.811806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:36.813167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:36.820962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 486/954] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Oct 12 12:23:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:36.826997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcb16b-65d4-478b-9735-7b9619883b4a None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:36.922385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:36.925240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dcb16b-65d4-478b-9735-7b9619883b4a None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:36.957188 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:36.957803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:36.958109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:36.958378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:36.968786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:36.970281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:36.984884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:36.986042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:36.987143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:36.987143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:36.987143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:36.987143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 469/955] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:36.992522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:37.059029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:37.093749 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119c089-db43-4da7-83ea-000409a66e68 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:37.126953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:37.127565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:37.127863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:37.128115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:37.128748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:37.129052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:37.130554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:37.142957 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 487/956] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:37.149104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:37.247707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ff08d-bd8a-4ef1-bfd7-74069a92959f None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:37.279889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:37.280503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:37.280783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:37.281052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:37.288892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:37.290425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:23:37.301336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:37.302497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:37.303623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 470/957] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:37.308423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:37.408963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223b2c7-fcf1-4ec7-bf33-b64bfd860795 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:37.433550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:37.441850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:37.442474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:37.442750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:37.443019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:37.443727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:37.444530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:37.445951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:37.462669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 488/958] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Oct 12 12:23:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:37.468569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:37.484238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:37.565070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93dced-5a49-4693-935c-e6237831329e None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:37.597939 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bebe85-e635-439b-be00-b5aa65a68b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:37.598558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bebe85-e635-439b-be00-b5aa65a68b26 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:37.598905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bebe85-e635-439b-be00-b5aa65a68b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:37.599172 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bebe85-e635-439b-be00-b5aa65a68b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:37.665133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bebe85-e635-439b-be00-b5aa65a68b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:37.666549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bebe85-e635-439b-be00-b5aa65a68b26 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:37.679387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:37.763296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 471/959] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Oct 12 12:23:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:23:37.769128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:37.867991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bed46-14bf-4e30-810a-140f2af839b4 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:37.902031 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:37.902646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:37.902924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:37.903202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:37.913127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:37.914577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:23:37.929449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:37.930574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:23:37.931641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:23:37.931641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:23:37.931641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:37.931641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 489/960] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:37.936339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bebe85-e635-439b-be00-b5aa65a68b26 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:38.032729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bebe85-e635-439b-be00-b5aa65a68b26 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:38.045368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:38.068229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:38.068229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:38.068229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:38.068229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:38.068596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:38.068596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:38.070208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:38.081501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 472/961] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Oct 12 12:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:38.087151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:38.123094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:38.185296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ae4ce-a5d9-411c-b291-d21799436879 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:38.216995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:38.217596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:38.217868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:38.218165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:38.224732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:38.227658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:23:38.238593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:38.239677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: ERROR keystone.server.flask.application Oct 12 12:23:38.240644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 490/962] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Oct 12 12:23:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:38.246192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:38.349737 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a414ae-8adc-4662-8ffb-faad4cf81c28 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:38.384654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29314659-34d0-4f05-8a0e-2c5ea50e2814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:38.385265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29314659-34d0-4f05-8a0e-2c5ea50e2814 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:38.385547 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29314659-34d0-4f05-8a0e-2c5ea50e2814 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:38.385817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29314659-34d0-4f05-8a0e-2c5ea50e2814 None admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:38.386531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29314659-34d0-4f05-8a0e-2c5ea50e2814 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:38.386841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29314659-34d0-4f05-8a0e-2c5ea50e2814 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:38.388289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29314659-34d0-4f05-8a0e-2c5ea50e2814 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:38.401837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 473/963] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Oct 12 12:23:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:38.408435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:38.514927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:38.520606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b38ff5-ab7a-4793-a24e-5d1bba4cc185 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:38.555263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326cbcc-cdfe-4b4b-8908-16e90c75c7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:38.555854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326cbcc-cdfe-4b4b-8908-16e90c75c7bb None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:38.556129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326cbcc-cdfe-4b4b-8908-16e90c75c7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:38.556398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326cbcc-cdfe-4b4b-8908-16e90c75c7bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:38.557119 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326cbcc-cdfe-4b4b-8908-16e90c75c7bb None admin] RBAC: Adding query filter params (role.id=7b4139f251c84cfdadc3cfd82f0f81c0, scope.project.id=e28d11b103f94db0ae7c772a2f086978, user.id=f35e7add10924dc49eb9d4a983c89eb3) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:38.557426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326cbcc-cdfe-4b4b-8908-16e90c75c7bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:38.558903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326cbcc-cdfe-4b4b-8908-16e90c75c7bb None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:38.564610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 491/964] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Oct 12 12:23:38 2022] GET /identity/v3/role_assignments?user.id=f35e7add10924dc49eb9d4a983c89eb3&scope.project.id=e28d11b103f94db0ae7c772a2f086978&role.id=7b4139f251c84cfdadc3cfd82f0f81c0 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:23:40.727954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15677268-3fbe-4526-8321-d080a48eb767 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:40.728486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15677268-3fbe-4526-8321-d080a48eb767 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:40.728486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15677268-3fbe-4526-8321-d080a48eb767 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:40.729831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 474/965] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:40.735090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af722e-fba0-478a-be40-9201923a9318 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:40.735465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af722e-fba0-478a-be40-9201923a9318 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:40.735628 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af722e-fba0-478a-be40-9201923a9318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:40.753919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a7af722e-fba0-478a-be40-9201923a9318 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:40.755114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a7af722e-fba0-478a-be40-9201923a9318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:40.818310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 492/966] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:41.087098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15677268-3fbe-4526-8321-d080a48eb767 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:41.118089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:41.186601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15677268-3fbe-4526-8321-d080a48eb767 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:41.220448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:41.221091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:41.221365 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:41.221632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:41.222226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:41.223565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:41.232295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:41.233441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:41.234332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 475/967] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Oct 12 12:23:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:41.239389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af722e-fba0-478a-be40-9201923a9318 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:41.327209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:41.336984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af722e-fba0-478a-be40-9201923a9318 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:41.369292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99f551-8cd3-41bf-92fd-da477c07e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:41.369917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99f551-8cd3-41bf-92fd-da477c07e59e None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:41.370223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99f551-8cd3-41bf-92fd-da477c07e59e None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:41.370491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99f551-8cd3-41bf-92fd-da477c07e59e None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:41.371106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99f551-8cd3-41bf-92fd-da477c07e59e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:41.371406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99f551-8cd3-41bf-92fd-da477c07e59e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:41.372744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99f551-8cd3-41bf-92fd-da477c07e59e None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:41.377381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 493/968] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Oct 12 12:23:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:41.383808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:41.476189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:41.482046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d950b6-bfa8-4cb2-90c3-fbb73273e775 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:41.515303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917660d9-5609-41c8-89e4-f562632580c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:41.515897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917660d9-5609-41c8-89e4-f562632580c2 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:41.516185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917660d9-5609-41c8-89e4-f562632580c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:41.516456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917660d9-5609-41c8-89e4-f562632580c2 None admin] PATH_INFO: `/v3/services` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:41.517076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917660d9-5609-41c8-89e4-f562632580c2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:41.517428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917660d9-5609-41c8-89e4-f562632580c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:41.518810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917660d9-5609-41c8-89e4-f562632580c2 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:41.523126 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 476/969] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Oct 12 12:23:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:43.658881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b805-2dc4-4192-a8f8-e2b83c3d034b None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:43.659407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b805-2dc4-4192-a8f8-e2b83c3d034b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:43.659407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3b805-2dc4-4192-a8f8-e2b83c3d034b None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:43.660760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 494/970] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:43.666093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a617-2b75-46e3-8e82-c7d28e526467 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:43.668919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a617-2b75-46e3-8e82-c7d28e526467 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:43.668919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a617-2b75-46e3-8e82-c7d28e526467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:43.684303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-28f5a617-2b75-46e3-8e82-c7d28e526467 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:43.685679 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-28f5a617-2b75-46e3-8e82-c7d28e526467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:43.751852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 477/971] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:44.017747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3b805-2dc4-4192-a8f8-e2b83c3d034b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:44.118331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3b805-2dc4-4192-a8f8-e2b83c3d034b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:44.150594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906c90f-519a-45fc-82ff-ad896fc520f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:44.151185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906c90f-519a-45fc-82ff-ad896fc520f9 None admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:44.151458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906c90f-519a-45fc-82ff-ad896fc520f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:44.151725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906c90f-519a-45fc-82ff-ad896fc520f9 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:44.152458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906c90f-519a-45fc-82ff-ad896fc520f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:44.153853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906c90f-519a-45fc-82ff-ad896fc520f9 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:44.261018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 495/972] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Oct 12 12:23:44 2022] POST /identity/v3/services => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:23:46.432545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37c87c-ce03-4649-94f7-d0fe72c8ba80 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:46.433055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37c87c-ce03-4649-94f7-d0fe72c8ba80 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:46.433055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37c87c-ce03-4649-94f7-d0fe72c8ba80 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:46.434398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 478/973] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:46.439613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e6ee-4cb1-434f-9824-b503ae2ecc27 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:46.440018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e6ee-4cb1-434f-9824-b503ae2ecc27 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:46.440161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e6ee-4cb1-434f-9824-b503ae2ecc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:46.458107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-da53e6ee-4cb1-434f-9824-b503ae2ecc27 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:46.459598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-da53e6ee-4cb1-434f-9824-b503ae2ecc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:46.468624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:46.530899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 496/974] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:46.799297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37c87c-ce03-4649-94f7-d0fe72c8ba80 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:46.889291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:46.899406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37c87c-ce03-4649-94f7-d0fe72c8ba80 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:46.932210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:46.932829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:46.933109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:46.933383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:46.934004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:46.935363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:46.943873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:46.950179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:46.952903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 479/975] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Oct 12 12:23:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:46.953873 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53e6ee-4cb1-434f-9824-b503ae2ecc27 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:47.049170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53e6ee-4cb1-434f-9824-b503ae2ecc27 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:47.085854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:47.087656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:47.087656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:47.087656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:47.087656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:47.087964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:47.089307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:47.094491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 497/976] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Oct 12 12:23:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:47.101009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:47.198194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a973f-da45-4094-abdf-8d62b84a7631 None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:47.230757 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91458e2-495f-468f-94bf-484e5b99cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:47.231352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91458e2-495f-468f-94bf-484e5b99cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:47.231633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91458e2-495f-468f-94bf-484e5b99cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:47.231907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91458e2-495f-468f-94bf-484e5b99cbb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:47.232602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91458e2-495f-468f-94bf-484e5b99cbb8 None admin] RBAC: Adding query filter params (interface=public, service_id=0be0455f791b4a50bc1d1ff9f6f07eca, region_id=RegionOne) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:47.232913 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91458e2-495f-468f-94bf-484e5b99cbb8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:47.234308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91458e2-495f-468f-94bf-484e5b99cbb8 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:47.239330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 480/977] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Oct 12 12:23:47 2022] GET /identity/v3/endpoints?service_id=0be0455f791b4a50bc1d1ff9f6f07eca&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:47.245459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:47.249284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:47.342547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d2bd-4c14-438e-a078-4500b9ac10e8 None admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:47.375583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13309c-111d-4b05-82eb-51340e18063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:47.376166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13309c-111d-4b05-82eb-51340e18063b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:47.376441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13309c-111d-4b05-82eb-51340e18063b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:47.376709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13309c-111d-4b05-82eb-51340e18063b None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:47.377297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13309c-111d-4b05-82eb-51340e18063b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:47.378658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13309c-111d-4b05-82eb-51340e18063b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:47.385263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 498/978] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Oct 12 12:23:47 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:23:49.509110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193c47-04cd-4ec5-a95a-ac52255a3b0b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:49.509624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193c47-04cd-4ec5-a95a-ac52255a3b0b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:49.509624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de193c47-04cd-4ec5-a95a-ac52255a3b0b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:49.510959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 481/979] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:49.516193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6004079-fbd8-4b44-89c8-7609f946fc60 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:49.516604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6004079-fbd8-4b44-89c8-7609f946fc60 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:49.516723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6004079-fbd8-4b44-89c8-7609f946fc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:49.535318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c6004079-fbd8-4b44-89c8-7609f946fc60 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:49.536495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c6004079-fbd8-4b44-89c8-7609f946fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:49.597318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:49.602942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 499/980] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:23:49.867818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de193c47-04cd-4ec5-a95a-ac52255a3b0b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:49.887533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:49.966590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de193c47-04cd-4ec5-a95a-ac52255a3b0b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:49.999687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:50.000303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:50.000582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:50.000859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] PATH_INFO: `/v3/services/placement` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:50.001433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:50.002818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:23:50.012110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:23:50.013337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application with Lock( Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: ERROR keystone.server.flask.application Oct 12 12:23:50.014225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 482/981] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Oct 12 12:23:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:23:50.019213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6004079-fbd8-4b44-89c8-7609f946fc60 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:50.119414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6004079-fbd8-4b44-89c8-7609f946fc60 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:50.151810 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50448bf5-c6cb-412e-8000-a318eb08af4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:50.152437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50448bf5-c6cb-412e-8000-a318eb08af4b None admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:50.152713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50448bf5-c6cb-412e-8000-a318eb08af4b None admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:50.152981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50448bf5-c6cb-412e-8000-a318eb08af4b None admin] PATH_INFO: `/v3/services` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:50.153594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50448bf5-c6cb-412e-8000-a318eb08af4b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:23:50.153892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50448bf5-c6cb-412e-8000-a318eb08af4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:50.155272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50448bf5-c6cb-412e-8000-a318eb08af4b None admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:50.160334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 500/982] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Oct 12 12:23:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:23:50.166727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:50.246588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:50.263918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cbb68-7010-46f2-bb29-7fc0a3f05cca None admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:50.295859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ba4c5-db55-47b4-abb2-29d05e0661d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:50.296450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ba4c5-db55-47b4-abb2-29d05e0661d1 None admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:50.296734 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ba4c5-db55-47b4-abb2-29d05e0661d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:50.297005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ba4c5-db55-47b4-abb2-29d05e0661d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:50.297657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ba4c5-db55-47b4-abb2-29d05e0661d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:50.299037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ba4c5-db55-47b4-abb2-29d05e0661d1 None admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:50.388666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 483/983] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:23:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:24:00.157247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=69 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:00.384215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:48.552708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f5b1-59f3-444e-9097-55823047ffde None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:48.553143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f5b1-59f3-444e-9097-55823047ffde None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:48.553228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93f5b1-59f3-444e-9097-55823047ffde None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:48.559875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57ae4-7521-4a60-aead-819eee0e2a49 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:48.560921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57ae4-7521-4a60-aead-819eee0e2a49 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:48.561625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57ae4-7521-4a60-aead-819eee0e2a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:48.567451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 501/984] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:29:48 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:48.654480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-acb57ae4-7521-4a60-aead-819eee0e2a49 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:48.655717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-acb57ae4-7521-4a60-aead-819eee0e2a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:48.782304 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 484/985] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:29:49.075293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d43ba1-bfa5-4d3a-bb71-8282a4876c6e None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:49.075745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d43ba1-bfa5-4d3a-bb71-8282a4876c6e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:49.076166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d43ba1-bfa5-4d3a-bb71-8282a4876c6e None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:49.078026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 502/986] 10.222.0.44 () {54 vars in 964 bytes} [Wed Oct 12 12:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:49.084610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170cf1-7018-4342-9789-d28980a1bd95 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:49.085156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170cf1-7018-4342-9789-d28980a1bd95 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:49.085582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98170cf1-7018-4342-9789-d28980a1bd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:49.172430 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-98170cf1-7018-4342-9789-d28980a1bd95 None None] MFA Rules not processed for user `db04575738b04f2d95803498e19554c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:49.173915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-98170cf1-7018-4342-9789-d28980a1bd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:49.196458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:49.272593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 485/987] 10.222.0.44 () {58 vars in 1061 bytes} [Wed Oct 12 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:49.281814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d43ba1-bfa5-4d3a-bb71-8282a4876c6e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:49.283839 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-98d43ba1-bfa5-4d3a-bb71-8282a4876c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:49.405692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d43ba1-bfa5-4d3a-bb71-8282a4876c6e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:49.508654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e809f2f-359c-44aa-a7e8-c36b76fab68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:49.509660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809f2f-359c-44aa-a7e8-c36b76fab68e service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:49.510314 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809f2f-359c-44aa-a7e8-c36b76fab68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:49.510833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809f2f-359c-44aa-a7e8-c36b76fab68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:49.513097 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4e809f2f-359c-44aa-a7e8-c36b76fab68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:49.580943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:49.642233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e809f2f-359c-44aa-a7e8-c36b76fab68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:49.643655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e809f2f-359c-44aa-a7e8-c36b76fab68e service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:49.664091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 503/988] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:52.121804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486031-f422-46be-ba07-5c9c7d68cfcc None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:52.122319 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486031-f422-46be-ba07-5c9c7d68cfcc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:52.122407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486031-f422-46be-ba07-5c9c7d68cfcc None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:52.123891 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 486/989] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:52.129062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5587f-042c-49f7-8025-bc3df9080905 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:52.129480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5587f-042c-49f7-8025-bc3df9080905 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:52.129637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5587f-042c-49f7-8025-bc3df9080905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:52.202564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d6a5587f-042c-49f7-8025-bc3df9080905 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:52.203780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d6a5587f-042c-49f7-8025-bc3df9080905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:52.301625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 504/990] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:29:52.585312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486031-f422-46be-ba07-5c9c7d68cfcc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:52.587368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e9486031-f422-46be-ba07-5c9c7d68cfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:52.689770 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486031-f422-46be-ba07-5c9c7d68cfcc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:52.760339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:52.762893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214bef7-21dd-4d95-8f4b-43b4e5f3b91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:52.763493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bef7-21dd-4d95-8f4b-43b4e5f3b91a service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:52.763773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bef7-21dd-4d95-8f4b-43b4e5f3b91a service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:52.764045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bef7-21dd-4d95-8f4b-43b4e5f3b91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:52.765659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6214bef7-21dd-4d95-8f4b-43b4e5f3b91a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:52.872372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214bef7-21dd-4d95-8f4b-43b4e5f3b91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:52.873712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214bef7-21dd-4d95-8f4b-43b4e5f3b91a service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:52.891640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 487/991] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:55.490796 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb4844-19d8-441b-97aa-45b3049e9e56 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:55.491294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb4844-19d8-441b-97aa-45b3049e9e56 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:55.491294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb4844-19d8-441b-97aa-45b3049e9e56 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:55.492583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 505/992] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:55.497866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d058-9671-4256-8e51-6d12b88b9380 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:55.498326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d058-9671-4256-8e51-6d12b88b9380 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:55.498466 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974d058-9671-4256-8e51-6d12b88b9380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:55.562815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3974d058-9671-4256-8e51-6d12b88b9380 None None] MFA Rules not processed for user `69e6e73e380845b897bd84a9478b968e`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:55.564016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3974d058-9671-4256-8e51-6d12b88b9380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:55.616154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:55.659777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 488/993] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:55.943366 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb4844-19d8-441b-97aa-45b3049e9e56 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:55.944884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-cfcb4844-19d8-441b-97aa-45b3049e9e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:55.956021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:56.048858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb4844-19d8-441b-97aa-45b3049e9e56 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:56.121378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da798c-33cc-4a8c-8928-5cc81846ba4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:56.121998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da798c-33cc-4a8c-8928-5cc81846ba4e service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:56.122277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da798c-33cc-4a8c-8928-5cc81846ba4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:56.122545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da798c-33cc-4a8c-8928-5cc81846ba4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:56.124122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-32da798c-33cc-4a8c-8928-5cc81846ba4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:56.187173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:56.234013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da798c-33cc-4a8c-8928-5cc81846ba4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:56.235377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da798c-33cc-4a8c-8928-5cc81846ba4e service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:56.253572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 506/994] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:59.162326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943ebfb-c0c6-45b1-91dd-422ad3aae266 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:59.162858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943ebfb-c0c6-45b1-91dd-422ad3aae266 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:59.162858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943ebfb-c0c6-45b1-91dd-422ad3aae266 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:59.164226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 489/995] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:59.170036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaae95-4916-4cb9-b8b6-08ec6076517b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:59.170436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaae95-4916-4cb9-b8b6-08ec6076517b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:59.170589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaae95-4916-4cb9-b8b6-08ec6076517b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:59.235940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-0faaae95-4916-4cb9-b8b6-08ec6076517b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:59.237140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0faaae95-4916-4cb9-b8b6-08ec6076517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:59.331672 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 507/996] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:29:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:59.612236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943ebfb-c0c6-45b1-91dd-422ad3aae266 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:59.613777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0943ebfb-c0c6-45b1-91dd-422ad3aae266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:59.715780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943ebfb-c0c6-45b1-91dd-422ad3aae266 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:59.787786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77efdab-ee3f-418c-9f6d-6d109fd7b2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:59.788383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77efdab-ee3f-418c-9f6d-6d109fd7b2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:59.788662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77efdab-ee3f-418c-9f6d-6d109fd7b2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:59.788932 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77efdab-ee3f-418c-9f6d-6d109fd7b2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:59.790561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e77efdab-ee3f-418c-9f6d-6d109fd7b2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:59.797300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:59.896612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77efdab-ee3f-418c-9f6d-6d109fd7b2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:59.897969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77efdab-ee3f-418c-9f6d-6d109fd7b2a7 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:59.915796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 490/997] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:02.997079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5249-69e0-4af7-aa8f-8ec86324e2b8 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:02.997634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5249-69e0-4af7-aa8f-8ec86324e2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:02.997634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f5249-69e0-4af7-aa8f-8ec86324e2b8 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:02.999011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 508/998] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:03.004287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a4dcd-787f-4fa1-bfc6-3c0da2b70e86 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:03.004700 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a4dcd-787f-4fa1-bfc6-3c0da2b70e86 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:03.004872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a4dcd-787f-4fa1-bfc6-3c0da2b70e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:03.077063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ec2a4dcd-787f-4fa1-bfc6-3c0da2b70e86 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:03.078376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ec2a4dcd-787f-4fa1-bfc6-3c0da2b70e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:03.141568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:03.184218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 491/999] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:03.465074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f5249-69e0-4af7-aa8f-8ec86324e2b8 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:03.466557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-519f5249-69e0-4af7-aa8f-8ec86324e2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:03.487058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:03.571079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f5249-69e0-4af7-aa8f-8ec86324e2b8 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:03.644130 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e5c5c-8d51-44f1-86f4-7f217c15fd3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:03.644731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e5c5c-8d51-44f1-86f4-7f217c15fd3e service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:03.645006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e5c5c-8d51-44f1-86f4-7f217c15fd3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:03.645270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e5c5c-8d51-44f1-86f4-7f217c15fd3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:03.646893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4f0e5c5c-8d51-44f1-86f4-7f217c15fd3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:03.720304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:03.757422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e5c5c-8d51-44f1-86f4-7f217c15fd3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:03.759254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e5c5c-8d51-44f1-86f4-7f217c15fd3e service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:03.778516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 509/1000] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:05.970753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e351-b471-4a1e-b9ab-19c8765d246e None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:05.971339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e351-b471-4a1e-b9ab-19c8765d246e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:05.971472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e351-b471-4a1e-b9ab-19c8765d246e None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:05.972838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 492/1001] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:05.978629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a839-bfff-4422-bdc9-6cfa012ce687 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:05.979041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a839-bfff-4422-bdc9-6cfa012ce687 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:05.979176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a839-bfff-4422-bdc9-6cfa012ce687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:06.044492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-41b7a839-bfff-4422-bdc9-6cfa012ce687 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:06.045698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-41b7a839-bfff-4422-bdc9-6cfa012ce687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:06.177347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 510/1002] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:06.462644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e351-b471-4a1e-b9ab-19c8765d246e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:06.464160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6123e351-b471-4a1e-b9ab-19c8765d246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:06.573751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e351-b471-4a1e-b9ab-19c8765d246e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:06.648184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66041d59-45f2-4183-9366-a0fd2e32cd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:06.648894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66041d59-45f2-4183-9366-a0fd2e32cd25 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:06.649827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66041d59-45f2-4183-9366-a0fd2e32cd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:06.650166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66041d59-45f2-4183-9366-a0fd2e32cd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:06.651751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-66041d59-45f2-4183-9366-a0fd2e32cd25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:06.668642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:06.760334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66041d59-45f2-4183-9366-a0fd2e32cd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:06.761671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66041d59-45f2-4183-9366-a0fd2e32cd25 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:06.779623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 493/1003] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:08.984649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11285d6-ff05-47c6-9014-8c69254fdc00 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:08.985146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11285d6-ff05-47c6-9014-8c69254fdc00 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:08.985146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11285d6-ff05-47c6-9014-8c69254fdc00 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:08.986581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 511/1004] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:08.991962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bbfe7-ed51-4951-b913-0cb7a1c1792c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:08.992386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bbfe7-ed51-4951-b913-0cb7a1c1792c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:08.992514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bbfe7-ed51-4951-b913-0cb7a1c1792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:09.058627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-436bbfe7-ed51-4951-b913-0cb7a1c1792c None None] MFA Rules not processed for user `69e6e73e380845b897bd84a9478b968e`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:09.059774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-436bbfe7-ed51-4951-b913-0cb7a1c1792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:09.127886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:09.158659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 494/1005] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:09.436449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11285d6-ff05-47c6-9014-8c69254fdc00 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:09.438046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a11285d6-ff05-47c6-9014-8c69254fdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:09.471230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:09.546932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11285d6-ff05-47c6-9014-8c69254fdc00 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:09.620598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe25f5e-2807-4358-b243-21594f122a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:09.621114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25f5e-2807-4358-b243-21594f122a2f service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:09.621389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25f5e-2807-4358-b243-21594f122a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:09.621657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe25f5e-2807-4358-b243-21594f122a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:09.623259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1fe25f5e-2807-4358-b243-21594f122a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:09.701033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:09.734346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe25f5e-2807-4358-b243-21594f122a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:09.735766 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe25f5e-2807-4358-b243-21594f122a2f service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:09.754815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 512/1006] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:12.263180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e6906-54c0-4578-a54e-8dc872e4fac2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:12.263711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e6906-54c0-4578-a54e-8dc872e4fac2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:12.263711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e6906-54c0-4578-a54e-8dc872e4fac2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:12.264994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 495/1007] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:12.270252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573db27c-cfec-43c9-bc1b-21f6de1cead1 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:12.270664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573db27c-cfec-43c9-bc1b-21f6de1cead1 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:12.270816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573db27c-cfec-43c9-bc1b-21f6de1cead1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:12.338087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-573db27c-cfec-43c9-bc1b-21f6de1cead1 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:12.339156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-573db27c-cfec-43c9-bc1b-21f6de1cead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:12.437157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 513/1008] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:12.721202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e6906-54c0-4578-a54e-8dc872e4fac2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:12.722785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-398e6906-54c0-4578-a54e-8dc872e4fac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:12.824977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e6906-54c0-4578-a54e-8dc872e4fac2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:12.898790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb86b7-30c9-4afd-8529-b62af3b2c684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:12.899391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb86b7-30c9-4afd-8529-b62af3b2c684 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:12.899663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb86b7-30c9-4afd-8529-b62af3b2c684 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:12.899930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb86b7-30c9-4afd-8529-b62af3b2c684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:12.901495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8bfb86b7-30c9-4afd-8529-b62af3b2c684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:12.927416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:13.008590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb86b7-30c9-4afd-8529-b62af3b2c684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:13.010482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb86b7-30c9-4afd-8529-b62af3b2c684 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:13.028987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 496/1009] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:15.216688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc052-b9f0-4350-8e54-7ef035c0e2e0 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:15.217216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc052-b9f0-4350-8e54-7ef035c0e2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:15.217216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc052-b9f0-4350-8e54-7ef035c0e2e0 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:15.218606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 514/1010] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:15.223901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f4108-1550-4c69-9f12-c86a59adf90b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:15.224300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f4108-1550-4c69-9f12-c86a59adf90b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:15.224448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f4108-1550-4c69-9f12-c86a59adf90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:15.289905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a77f4108-1550-4c69-9f12-c86a59adf90b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:15.291663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a77f4108-1550-4c69-9f12-c86a59adf90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:15.367697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:15.390136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 497/1011] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:15.674692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cc052-b9f0-4350-8e54-7ef035c0e2e0 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:15.676251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f43cc052-b9f0-4350-8e54-7ef035c0e2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:15.716264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:15.779822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cc052-b9f0-4350-8e54-7ef035c0e2e0 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:15.852230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac457c0a-2a97-43d0-bed2-dfdb03e939fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:15.853413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac457c0a-2a97-43d0-bed2-dfdb03e939fd service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:15.853692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac457c0a-2a97-43d0-bed2-dfdb03e939fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:15.853998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac457c0a-2a97-43d0-bed2-dfdb03e939fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:15.855850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ac457c0a-2a97-43d0-bed2-dfdb03e939fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:15.938915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:15.964451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac457c0a-2a97-43d0-bed2-dfdb03e939fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:15.966125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac457c0a-2a97-43d0-bed2-dfdb03e939fd service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:15.985109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 515/1012] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:18.729278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e4cfb-2f79-4ead-a36a-e1d7b0e970a9 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:18.729808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e4cfb-2f79-4ead-a36a-e1d7b0e970a9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:18.729808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e4cfb-2f79-4ead-a36a-e1d7b0e970a9 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:18.731447 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 498/1013] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:18.736802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf679dea-90b6-465d-96d5-cb49dda88f9e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:18.737241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf679dea-90b6-465d-96d5-cb49dda88f9e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:18.737335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf679dea-90b6-465d-96d5-cb49dda88f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:18.804042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-cf679dea-90b6-465d-96d5-cb49dda88f9e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:18.805250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-cf679dea-90b6-465d-96d5-cb49dda88f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:18.903805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 516/1014] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:19.204317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdfb0-7ccd-4357-b9e7-4fda93a9f015 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:19.204593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdfb0-7ccd-4357-b9e7-4fda93a9f015 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:19.207965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdfb0-7ccd-4357-b9e7-4fda93a9f015 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:19.207965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 499/1015] 10.222.0.44 () {54 vars in 964 bytes} [Wed Oct 12 12:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:19.212154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574c333-3b64-4fc8-b3f3-192b04a32fb0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:19.212460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574c333-3b64-4fc8-b3f3-192b04a32fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:19.212754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574c333-3b64-4fc8-b3f3-192b04a32fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:19.268626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:19.289473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-f574c333-3b64-4fc8-b3f3-192b04a32fb0 None None] MFA Rules not processed for user `db04575738b04f2d95803498e19554c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:19.291193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f574c333-3b64-4fc8-b3f3-192b04a32fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:19.366252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 517/1016] 10.222.0.44 () {58 vars in 1061 bytes} [Wed Oct 12 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:19.375718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccdfb0-7ccd-4357-b9e7-4fda93a9f015 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:19.377234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-adccdfb0-7ccd-4357-b9e7-4fda93a9f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:19.479056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccdfb0-7ccd-4357-b9e7-4fda93a9f015 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:19.552139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff32470-445c-4fd8-ba1f-1bc7787e7dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:19.552727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff32470-445c-4fd8-ba1f-1bc7787e7dee service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:19.553003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff32470-445c-4fd8-ba1f-1bc7787e7dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:19.553270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff32470-445c-4fd8-ba1f-1bc7787e7dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:19.554874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9ff32470-445c-4fd8-ba1f-1bc7787e7dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:19.592566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:19.678117 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff32470-445c-4fd8-ba1f-1bc7787e7dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:19.679484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff32470-445c-4fd8-ba1f-1bc7787e7dee service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:19.697872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 500/1017] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:24.668036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ae5e0-6b19-4938-b313-9f2d7b8bc402 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:24.668634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ae5e0-6b19-4938-b313-9f2d7b8bc402 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:24.668634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ae5e0-6b19-4938-b313-9f2d7b8bc402 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:24.669928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 518/1018] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:24.677377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99c273-29ca-4822-90dd-000939089dc0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:24.677839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99c273-29ca-4822-90dd-000939089dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:24.677919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99c273-29ca-4822-90dd-000939089dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:24.746607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-df99c273-29ca-4822-90dd-000939089dc0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:24.748315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-df99c273-29ca-4822-90dd-000939089dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:24.836389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:24.847331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 501/1019] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:30:25.137162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ae5e0-6b19-4938-b313-9f2d7b8bc402 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:25.139065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5a4ae5e0-6b19-4938-b313-9f2d7b8bc402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:25.246964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ae5e0-6b19-4938-b313-9f2d7b8bc402 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:25.286042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:25.324455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab258d7-4992-4d7a-8e8f-9e75784a5213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:25.325107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab258d7-4992-4d7a-8e8f-9e75784a5213 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:25.325384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab258d7-4992-4d7a-8e8f-9e75784a5213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:25.325668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab258d7-4992-4d7a-8e8f-9e75784a5213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:25.327243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-dab258d7-4992-4d7a-8e8f-9e75784a5213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:25.437666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab258d7-4992-4d7a-8e8f-9e75784a5213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:25.439022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab258d7-4992-4d7a-8e8f-9e75784a5213 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:25.457253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 519/1020] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:28.643302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c59c5-8df9-4191-b280-a71903b1cdfb None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:28.643802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c59c5-8df9-4191-b280-a71903b1cdfb None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:28.643802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c59c5-8df9-4191-b280-a71903b1cdfb None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:28.645168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 502/1021] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:28.651169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ad2a7-6241-43c8-a04b-c801489847d8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:28.651600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ad2a7-6241-43c8-a04b-c801489847d8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:28.651722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ad2a7-6241-43c8-a04b-c801489847d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:28.719032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ea2ad2a7-6241-43c8-a04b-c801489847d8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:28.720239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ea2ad2a7-6241-43c8-a04b-c801489847d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:28.740418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:28.822824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 520/1022] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:29.103649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c59c5-8df9-4191-b280-a71903b1cdfb None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:29.105202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-375c59c5-8df9-4191-b280-a71903b1cdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:29.211492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c59c5-8df9-4191-b280-a71903b1cdfb None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:29.302577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee543f-d8ab-4fbd-98be-1a0452f9fe8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:29.303181 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee543f-d8ab-4fbd-98be-1a0452f9fe8d service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:29.303459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee543f-d8ab-4fbd-98be-1a0452f9fe8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:29.303727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee543f-d8ab-4fbd-98be-1a0452f9fe8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:29.305307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3dee543f-d8ab-4fbd-98be-1a0452f9fe8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:29.357301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:29.430418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee543f-d8ab-4fbd-98be-1a0452f9fe8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:29.431882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee543f-d8ab-4fbd-98be-1a0452f9fe8d service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:29.451919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 503/1023] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:34.013627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518734-9289-4ee3-85bc-2080c77b00f2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:34.014189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518734-9289-4ee3-85bc-2080c77b00f2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:34.014189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea518734-9289-4ee3-85bc-2080c77b00f2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:34.015544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 521/1024] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:34.022672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741952-fa2a-401c-bf6e-d535600e408c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:34.023131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741952-fa2a-401c-bf6e-d535600e408c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:34.023131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741952-fa2a-401c-bf6e-d535600e408c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:34.107613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a2741952-fa2a-401c-bf6e-d535600e408c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:34.108982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a2741952-fa2a-401c-bf6e-d535600e408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:34.229160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:34.231960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 504/1025] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:34.517210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea518734-9289-4ee3-85bc-2080c77b00f2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:34.518817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ea518734-9289-4ee3-85bc-2080c77b00f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:34.634354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea518734-9289-4ee3-85bc-2080c77b00f2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:34.689405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:34.719615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881473c-70db-4444-912f-102017932ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:34.720222 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881473c-70db-4444-912f-102017932ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:34.720495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881473c-70db-4444-912f-102017932ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:34.720761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881473c-70db-4444-912f-102017932ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:34.722403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8881473c-70db-4444-912f-102017932ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:34.838016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881473c-70db-4444-912f-102017932ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:34.839440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881473c-70db-4444-912f-102017932ac6 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:34.858539 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 522/1026] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:40.230491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cca2e-75c2-41dd-924d-67a35ff71e32 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:40.230491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cca2e-75c2-41dd-924d-67a35ff71e32 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:40.230491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cca2e-75c2-41dd-924d-67a35ff71e32 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:40.232121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 505/1027] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:40.237542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741e7ab-65a4-45e7-b77a-5d1e6affbca3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:40.238501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741e7ab-65a4-45e7-b77a-5d1e6affbca3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:40.239008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741e7ab-65a4-45e7-b77a-5d1e6affbca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:40.475632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9741e7ab-65a4-45e7-b77a-5d1e6affbca3 None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:40.476999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9741e7ab-65a4-45e7-b77a-5d1e6affbca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:40.515755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:40.705381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 523/1028] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:30:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:45.349608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eebae-d061-4047-9844-c17830f460ee None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:45.350685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eebae-d061-4047-9844-c17830f460ee None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:45.351117 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31eebae-d061-4047-9844-c17830f460ee None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:45.353094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 506/1029] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:45.358593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff18e39-bcc9-4035-8544-a0ca9bcad888 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:45.359026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff18e39-bcc9-4035-8544-a0ca9bcad888 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:45.359245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff18e39-bcc9-4035-8544-a0ca9bcad888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:45.447342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-fff18e39-bcc9-4035-8544-a0ca9bcad888 None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:45.448558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fff18e39-bcc9-4035-8544-a0ca9bcad888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:45.523274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:45.564662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 524/1030] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:46.861298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80011eb2-55e3-4dd8-af01-ef1dd47320da None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:46.861818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80011eb2-55e3-4dd8-af01-ef1dd47320da None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:46.861818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80011eb2-55e3-4dd8-af01-ef1dd47320da None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:46.863473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 507/1031] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:46.868266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e27f-fe3f-4dae-b81a-0e8b98bc9b9b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:46.868654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e27f-fe3f-4dae-b81a-0e8b98bc9b9b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:46.868926 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e27f-fe3f-4dae-b81a-0e8b98bc9b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:46.944693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b6d7e27f-fe3f-4dae-b81a-0e8b98bc9b9b None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:46.945880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b6d7e27f-fe3f-4dae-b81a-0e8b98bc9b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:47.052109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 525/1032] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:50.597683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406435a-4f83-4931-8022-5d5fe8eed2bc None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.598126 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406435a-4f83-4931-8022-5d5fe8eed2bc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.600187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406435a-4f83-4931-8022-5d5fe8eed2bc None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.605272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 508/1033] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:50.617992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5eebd-3427-4d49-afbf-759aeb799e82 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.617992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5eebd-3427-4d49-afbf-759aeb799e82 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.617992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5eebd-3427-4d49-afbf-759aeb799e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.658960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:50.696906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b3a5eebd-3427-4d49-afbf-759aeb799e82 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:50.703689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b3a5eebd-3427-4d49-afbf-759aeb799e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:50.825980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 526/1034] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.130866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b2d3b-7dd4-4cfd-b720-a01e5db3cff7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.131019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b2d3b-7dd4-4cfd-b720-a01e5db3cff7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.131268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b2d3b-7dd4-4cfd-b720-a01e5db3cff7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.132788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 509/1035] 10.222.0.44 () {54 vars in 961 bytes} [Wed Oct 12 12:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:51.143881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6fef0-8075-43e1-afdb-421ea948aac4 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.144315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6fef0-8075-43e1-afdb-421ea948aac4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.144661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6fef0-8075-43e1-afdb-421ea948aac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.234773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-6bc6fef0-8075-43e1-afdb-421ea948aac4 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.236054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6bc6fef0-8075-43e1-afdb-421ea948aac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.294476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:51.366848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 527/1036] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:30:51.375612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b2d3b-7dd4-4cfd-b720-a01e5db3cff7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.377116 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e79b2d3b-7dd4-4cfd-b720-a01e5db3cff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.527452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b2d3b-7dd4-4cfd-b720-a01e5db3cff7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.603049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:51.644329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97096d-3d7c-44c0-a22c-4b314d51a5e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.645078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97096d-3d7c-44c0-a22c-4b314d51a5e1 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.645356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97096d-3d7c-44c0-a22c-4b314d51a5e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.645623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97096d-3d7c-44c0-a22c-4b314d51a5e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.647211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ac97096d-3d7c-44c0-a22c-4b314d51a5e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.748775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97096d-3d7c-44c0-a22c-4b314d51a5e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.750375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97096d-3d7c-44c0-a22c-4b314d51a5e1 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.769270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 510/1037] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:54.131319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b251f-bbf5-46b0-ab93-c6450b278aef None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:54.131815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b251f-bbf5-46b0-ab93-c6450b278aef None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:54.131815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b251f-bbf5-46b0-ab93-c6450b278aef None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:54.133121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 528/1038] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:54.138542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6887-371c-4a72-8311-31bf972d6214 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:54.138959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6887-371c-4a72-8311-31bf972d6214 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:54.139089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace6887-371c-4a72-8311-31bf972d6214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:54.236350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1ace6887-371c-4a72-8311-31bf972d6214 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:54.237613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1ace6887-371c-4a72-8311-31bf972d6214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:54.278332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:54.340574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 511/1039] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:54.628964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b251f-bbf5-46b0-ab93-c6450b278aef None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:54.630504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bd9b251f-bbf5-46b0-ab93-c6450b278aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:54.811368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b251f-bbf5-46b0-ab93-c6450b278aef None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:54.823670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:54.938159 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319838e4-1840-4920-9535-e5e41466e1c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:54.938675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319838e4-1840-4920-9535-e5e41466e1c1 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:54.938944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319838e4-1840-4920-9535-e5e41466e1c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:54.939207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319838e4-1840-4920-9535-e5e41466e1c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:54.940794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-319838e4-1840-4920-9535-e5e41466e1c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:55.035387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319838e4-1840-4920-9535-e5e41466e1c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:55.036974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319838e4-1840-4920-9535-e5e41466e1c1 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:55.055039 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 529/1040] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:57.618840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0753d-d531-45a2-a142-e819ded48991 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:57.619355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0753d-d531-45a2-a142-e819ded48991 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:57.619355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0753d-d531-45a2-a142-e819ded48991 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:57.620653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 512/1041] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:57.626151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f060e-825d-44a5-b9a4-739e986fc88f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:57.626596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f060e-825d-44a5-b9a4-739e986fc88f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:57.626689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f060e-825d-44a5-b9a4-739e986fc88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:57.644039 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:57.697320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-943f060e-825d-44a5-b9a4-739e986fc88f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:57.698574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-943f060e-825d-44a5-b9a4-739e986fc88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:57.809017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 530/1042] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:58.107772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabaaad-29aa-461e-9357-3353f28411be None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.108177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabaaad-29aa-461e-9357-3353f28411be None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.108584 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabaaad-29aa-461e-9357-3353f28411be None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.110174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 513/1043] 10.222.0.44 () {54 vars in 961 bytes} [Wed Oct 12 12:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:58.115655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1658ba-60c1-408a-b8af-a29bbd954db3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.116106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1658ba-60c1-408a-b8af-a29bbd954db3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.127789 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1658ba-60c1-408a-b8af-a29bbd954db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.196935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-fb1658ba-60c1-408a-b8af-a29bbd954db3 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:58.198159 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fb1658ba-60c1-408a-b8af-a29bbd954db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:58.235846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:58.322830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 531/1044] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Oct 12 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:58.330970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabaaad-29aa-461e-9357-3353f28411be None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:58.332546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7fabaaad-29aa-461e-9357-3353f28411be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:58.496503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabaaad-29aa-461e-9357-3353f28411be None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:58.519710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:58.631204 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40532b24-9a9a-4f6a-8bf8-99ef1688ae16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:58.631863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40532b24-9a9a-4f6a-8bf8-99ef1688ae16 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.632037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40532b24-9a9a-4f6a-8bf8-99ef1688ae16 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.632463 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40532b24-9a9a-4f6a-8bf8-99ef1688ae16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.634220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-40532b24-9a9a-4f6a-8bf8-99ef1688ae16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:58.734104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40532b24-9a9a-4f6a-8bf8-99ef1688ae16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:58.735702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40532b24-9a9a-4f6a-8bf8-99ef1688ae16 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:58.756018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 514/1045] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:58.813490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c9869-a885-4a1a-9535-07c80e3a1ada None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.814061 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c9869-a885-4a1a-9535-07c80e3a1ada None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.814156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c9869-a885-4a1a-9535-07c80e3a1ada None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.815504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 532/1046] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:58.820914 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf2eb2-4b83-4e3e-b849-2a16ec06ccd5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.821274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf2eb2-4b83-4e3e-b849-2a16ec06ccd5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.821613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf2eb2-4b83-4e3e-b849-2a16ec06ccd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.854683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:58.891465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b8cf2eb2-4b83-4e3e-b849-2a16ec06ccd5 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:58.892720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b8cf2eb2-4b83-4e3e-b849-2a16ec06ccd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:59.026215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 515/1047] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:59.351673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee401b2-dbf4-4566-8b9a-6fa045426f74 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.352080 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee401b2-dbf4-4566-8b9a-6fa045426f74 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.352474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee401b2-dbf4-4566-8b9a-6fa045426f74 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.354032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 533/1048] 10.222.0.44 () {54 vars in 963 bytes} [Wed Oct 12 12:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:59.359256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38729b1-0a5a-4778-b885-ed049c2f5d91 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.359542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38729b1-0a5a-4778-b885-ed049c2f5d91 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.359817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38729b1-0a5a-4778-b885-ed049c2f5d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.440145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a38729b1-0a5a-4778-b885-ed049c2f5d91 None None] MFA Rules not processed for user `8238fbc3012844ea8b2b5ab13c8ea5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:59.441353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a38729b1-0a5a-4778-b885-ed049c2f5d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:59.492598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:59.522741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 516/1049] 10.222.0.44 () {58 vars in 1060 bytes} [Wed Oct 12 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:59.530949 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee401b2-dbf4-4566-8b9a-6fa045426f74 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:59.532648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4ee401b2-dbf4-4566-8b9a-6fa045426f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:59.653105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee401b2-dbf4-4566-8b9a-6fa045426f74 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:59.701387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:59.738474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b5680-6870-4aee-8495-e276e63bbf50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:59.739181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5680-6870-4aee-8495-e276e63bbf50 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.739554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5680-6870-4aee-8495-e276e63bbf50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.739917 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b5680-6870-4aee-8495-e276e63bbf50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.741764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-596b5680-6870-4aee-8495-e276e63bbf50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:59.857112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b5680-6870-4aee-8495-e276e63bbf50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:59.857112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b5680-6870-4aee-8495-e276e63bbf50 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:59.880449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 534/1050] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:01.143256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44f232-c708-4055-bfc8-2d4f47c2faa2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:01.143773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44f232-c708-4055-bfc8-2d4f47c2faa2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:01.147892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44f232-c708-4055-bfc8-2d4f47c2faa2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:01.147892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 517/1051] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:01.150584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af219-de7d-4a7d-ad0e-93cd6ab950e5 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:01.150987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af219-de7d-4a7d-ad0e-93cd6ab950e5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:01.151141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97af219-de7d-4a7d-ad0e-93cd6ab950e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:01.219797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b97af219-de7d-4a7d-ad0e-93cd6ab950e5 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:01.221025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b97af219-de7d-4a7d-ad0e-93cd6ab950e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:01.242619 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:01.320402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 535/1052] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:01.611229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44f232-c708-4055-bfc8-2d4f47c2faa2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:01.612869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fa44f232-c708-4055-bfc8-2d4f47c2faa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:01.765468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44f232-c708-4055-bfc8-2d4f47c2faa2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:01.818674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:01.883055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582fecd0-7332-4d95-a3cd-6f15383361c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:01.884174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fecd0-7332-4d95-a3cd-6f15383361c1 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:01.884449 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fecd0-7332-4d95-a3cd-6f15383361c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:01.884717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fecd0-7332-4d95-a3cd-6f15383361c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:01.886663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-582fecd0-7332-4d95-a3cd-6f15383361c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:01.982418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582fecd0-7332-4d95-a3cd-6f15383361c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:01.984004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582fecd0-7332-4d95-a3cd-6f15383361c1 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:02.003722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 518/1053] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:04.286345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a272bc-7cba-4dc7-a51d-053beab18460 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.286894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a272bc-7cba-4dc7-a51d-053beab18460 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.286894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a272bc-7cba-4dc7-a51d-053beab18460 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.288358 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 536/1054] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:04.293981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00b5d7-1cd1-4a7e-8914-398e72392dec None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.294408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00b5d7-1cd1-4a7e-8914-398e72392dec None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.294521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00b5d7-1cd1-4a7e-8914-398e72392dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.362748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8f00b5d7-1cd1-4a7e-8914-398e72392dec None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:04.363852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8f00b5d7-1cd1-4a7e-8914-398e72392dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:04.371483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:04.466302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 519/1055] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:04.751574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a272bc-7cba-4dc7-a51d-053beab18460 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:04.753233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-29a272bc-7cba-4dc7-a51d-053beab18460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:04.910750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a272bc-7cba-4dc7-a51d-053beab18460 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:04.915345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:05.028803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3547340-bf95-4c92-b889-f3f3154d1d2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:05.029681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3547340-bf95-4c92-b889-f3f3154d1d2c service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:05.030094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3547340-bf95-4c92-b889-f3f3154d1d2c service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:05.030460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3547340-bf95-4c92-b889-f3f3154d1d2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:05.032228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b3547340-bf95-4c92-b889-f3f3154d1d2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:05.127975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3547340-bf95-4c92-b889-f3f3154d1d2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:05.129547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3547340-bf95-4c92-b889-f3f3154d1d2c service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:05.148682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 537/1056] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:07.471767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b4ac-c1ad-4e59-b8b0-b3ef655ba8c7 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:07.472374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b4ac-c1ad-4e59-b8b0-b3ef655ba8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:07.472374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b4ac-c1ad-4e59-b8b0-b3ef655ba8c7 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:07.478142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 520/1057] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:07.479353 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80dd9b-7558-4657-959e-6b54c87eb04b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:07.479816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80dd9b-7558-4657-959e-6b54c87eb04b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:07.479816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80dd9b-7558-4657-959e-6b54c87eb04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:07.497217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:07.573415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-dc80dd9b-7558-4657-959e-6b54c87eb04b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:07.574581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-dc80dd9b-7558-4657-959e-6b54c87eb04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:07.675414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 538/1058] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:07.970917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2b4ac-c1ad-4e59-b8b0-b3ef655ba8c7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:07.972443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-09e2b4ac-c1ad-4e59-b8b0-b3ef655ba8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:08.127371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:08.138188 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2b4ac-c1ad-4e59-b8b0-b3ef655ba8c7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:08.274978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedf970-7516-46b9-8550-17f403a2c8f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:08.275810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedf970-7516-46b9-8550-17f403a2c8f8 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:08.276174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedf970-7516-46b9-8550-17f403a2c8f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:08.276557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedf970-7516-46b9-8550-17f403a2c8f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:08.279056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ebedf970-7516-46b9-8550-17f403a2c8f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:08.377605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedf970-7516-46b9-8550-17f403a2c8f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:08.379219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedf970-7516-46b9-8550-17f403a2c8f8 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:08.389450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:08.397513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 521/1059] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:10.717176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c9ce4-5e30-4b69-9358-6866a8d028a5 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.717732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c9ce4-5e30-4b69-9358-6866a8d028a5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.717829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c9ce4-5e30-4b69-9358-6866a8d028a5 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.719292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 539/1060] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:10.724958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca1d2-65a9-4d20-9280-99556fb56de9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.725509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca1d2-65a9-4d20-9280-99556fb56de9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.725509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca1d2-65a9-4d20-9280-99556fb56de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.802055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-00eca1d2-65a9-4d20-9280-99556fb56de9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:10.803274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-00eca1d2-65a9-4d20-9280-99556fb56de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:10.908126 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 522/1061] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:11.284633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c9ce4-5e30-4b69-9358-6866a8d028a5 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:11.286203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-318c9ce4-5e30-4b69-9358-6866a8d028a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:11.373022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:11.437476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c9ce4-5e30-4b69-9358-6866a8d028a5 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:11.558196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab63388-ee5e-44fe-b500-29548cc18002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:11.558796 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab63388-ee5e-44fe-b500-29548cc18002 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:11.559071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab63388-ee5e-44fe-b500-29548cc18002 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:11.559336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab63388-ee5e-44fe-b500-29548cc18002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:11.560913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5ab63388-ee5e-44fe-b500-29548cc18002 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:11.619935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:11.659816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab63388-ee5e-44fe-b500-29548cc18002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:11.661395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab63388-ee5e-44fe-b500-29548cc18002 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:11.679624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 540/1062] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:31:13.820874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec456b8-2528-4665-a2ad-8c1f5840fc00 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:13.822313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec456b8-2528-4665-a2ad-8c1f5840fc00 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:13.822313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec456b8-2528-4665-a2ad-8c1f5840fc00 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:13.823436 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 523/1063] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:13.828897 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d74b52-aeae-46b0-bc61-f99bf4d321b9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:13.829327 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d74b52-aeae-46b0-bc61-f99bf4d321b9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:13.829439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d74b52-aeae-46b0-bc61-f99bf4d321b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:13.895509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-37d74b52-aeae-46b0-bc61-f99bf4d321b9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:13.896698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-37d74b52-aeae-46b0-bc61-f99bf4d321b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:13.993900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 541/1064] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:14.280809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec456b8-2528-4665-a2ad-8c1f5840fc00 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:14.282400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7ec456b8-2528-4665-a2ad-8c1f5840fc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:14.360116 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:14.430037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec456b8-2528-4665-a2ad-8c1f5840fc00 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:14.542536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e324ca-62b3-47e0-b122-60d330cfdf26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:14.543143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e324ca-62b3-47e0-b122-60d330cfdf26 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:14.543417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e324ca-62b3-47e0-b122-60d330cfdf26 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:14.543686 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e324ca-62b3-47e0-b122-60d330cfdf26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:14.545284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b6e324ca-62b3-47e0-b122-60d330cfdf26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:14.583157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:14.635542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e324ca-62b3-47e0-b122-60d330cfdf26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:14.637119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e324ca-62b3-47e0-b122-60d330cfdf26 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:14.655961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 524/1065] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:15.320221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678c3b1-8bbe-4580-bb44-cb359c372f33 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.320753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678c3b1-8bbe-4580-bb44-cb359c372f33 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.320753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678c3b1-8bbe-4580-bb44-cb359c372f33 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.322588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 542/1066] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:15.328065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada6a8e-dab4-4b17-8065-e62722d6cd0c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.328484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada6a8e-dab4-4b17-8065-e62722d6cd0c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.328623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada6a8e-dab4-4b17-8065-e62722d6cd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.396305 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7ada6a8e-dab4-4b17-8065-e62722d6cd0c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:15.397521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7ada6a8e-dab4-4b17-8065-e62722d6cd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:15.495817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 525/1067] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:15.793236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4bd-a343-471b-90cb-e8d8c597a938 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.793592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4bd-a343-471b-90cb-e8d8c597a938 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.793958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b4bd-a343-471b-90cb-e8d8c597a938 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.795354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 543/1068] 10.222.0.44 () {54 vars in 962 bytes} [Wed Oct 12 12:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:15.800799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a87e1-2bdb-403c-88d8-84f9b0291e3f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.801097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a87e1-2bdb-403c-88d8-84f9b0291e3f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.801376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a87e1-2bdb-403c-88d8-84f9b0291e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.818049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:15.878951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4e0a87e1-2bdb-403c-88d8-84f9b0291e3f None None] MFA Rules not processed for user `70cbcaeb843e470781661f00ec91bffd`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:15.880147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4e0a87e1-2bdb-403c-88d8-84f9b0291e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:15.955453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 526/1069] 10.222.0.44 () {58 vars in 1059 bytes} [Wed Oct 12 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:15.964528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b4bd-a343-471b-90cb-e8d8c597a938 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:15.966158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7805b4bd-a343-471b-90cb-e8d8c597a938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:15.997884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:16.077234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b4bd-a343-471b-90cb-e8d8c597a938 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:16.150998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f1464-e64a-447c-8c89-2f30ecd680a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cbcaeb843e470781661f00ec91bffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:16.151601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f1464-e64a-447c-8c89-2f30ecd680a3 service glance] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:16.151879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f1464-e64a-447c-8c89-2f30ecd680a3 service glance] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:16.152147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f1464-e64a-447c-8c89-2f30ecd680a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:16.153841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9b9f1464-e64a-447c-8c89-2f30ecd680a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:16.230566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:16.263146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f1464-e64a-447c-8c89-2f30ecd680a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:16.264565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f1464-e64a-447c-8c89-2f30ecd680a3 service glance] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:16.282323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 544/1070] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:16.940617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db7b5f-e977-4b90-b210-f4bc92afe350 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:16.941206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db7b5f-e977-4b90-b210-f4bc92afe350 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:16.941206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db7b5f-e977-4b90-b210-f4bc92afe350 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:16.942602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 527/1071] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:16.948595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d92d-1b76-419d-9800-533867626f15 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:16.949131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d92d-1b76-419d-9800-533867626f15 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:16.949131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d92d-1b76-419d-9800-533867626f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.019131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-6978d92d-1b76-419d-9800-533867626f15 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:17.020347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6978d92d-1b76-419d-9800-533867626f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.120882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 545/1072] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:17.157141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928400f-552f-45c2-bb70-a2323108c886 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.157452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928400f-552f-45c2-bb70-a2323108c886 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.157729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928400f-552f-45c2-bb70-a2323108c886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.226424 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7928400f-552f-45c2-bb70-a2323108c886 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:17.228209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7928400f-552f-45c2-bb70-a2323108c886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.285375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:17.316483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 528/1073] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:17.358909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f148-ab13-4672-912d-4465fb46eddd None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.359263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f148-ab13-4672-912d-4465fb46eddd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.359600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834f148-ab13-4672-912d-4465fb46eddd None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.361018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 546/1074] 10.222.0.44 () {54 vars in 943 bytes} [Wed Oct 12 12:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:17.367152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c17cfb-7150-4c01-89fb-4a1ba2b76f08 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.367438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c17cfb-7150-4c01-89fb-4a1ba2b76f08 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.367715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c17cfb-7150-4c01-89fb-4a1ba2b76f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.418281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f148-ab13-4672-912d-4465fb46eddd None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:17.419759 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4834f148-ab13-4672-912d-4465fb46eddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.447246 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-41c17cfb-7150-4c01-89fb-4a1ba2b76f08 None None] MFA Rules not processed for user `36af83e8ee4048d78d2db8784f6347df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:17.448421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-41c17cfb-7150-4c01-89fb-4a1ba2b76f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.460935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:17.554894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 529/1075] 10.222.0.44 () {58 vars in 1040 bytes} [Wed Oct 12 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:17.564677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c17cfb-7150-4c01-89fb-4a1ba2b76f08 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:17.573441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834f148-ab13-4672-912d-4465fb46eddd None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:17.574803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c17cfb-7150-4c01-89fb-4a1ba2b76f08 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:17.599987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:17.682599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5a68e-9cef-4a3b-a584-c59002f48ce6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:17.683206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5a68e-9cef-4a3b-a584-c59002f48ce6 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.683480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5a68e-9cef-4a3b-a584-c59002f48ce6 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.683745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5a68e-9cef-4a3b-a584-c59002f48ce6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.685399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d4f5a68e-9cef-4a3b-a584-c59002f48ce6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.690954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:17.695219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead5a0f-e4af-4934-aeed-d0186704fcb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:17.697813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead5a0f-e4af-4934-aeed-d0186704fcb0 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.697813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead5a0f-e4af-4934-aeed-d0186704fcb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.697813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead5a0f-e4af-4934-aeed-d0186704fcb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.699055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bead5a0f-e4af-4934-aeed-d0186704fcb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.771820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5a68e-9cef-4a3b-a584-c59002f48ce6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:17.773538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5a68e-9cef-4a3b-a584-c59002f48ce6 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:17.782413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 530/1076] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:31:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 12:31:17.795489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead5a0f-e4af-4934-aeed-d0186704fcb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:17.797146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead5a0f-e4af-4934-aeed-d0186704fcb0 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:17.816012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 547/1077] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:31:20.112896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf724-e936-4d60-9db2-2d4876712c84 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.113507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf724-e936-4d60-9db2-2d4876712c84 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.113507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf724-e936-4d60-9db2-2d4876712c84 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.117420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 531/1078] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:20.120960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681596b-1c75-46a7-ae8e-3190c037d10e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.121393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681596b-1c75-46a7-ae8e-3190c037d10e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.121515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681596b-1c75-46a7-ae8e-3190c037d10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.197479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2681596b-1c75-46a7-ae8e-3190c037d10e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:20.199127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2681596b-1c75-46a7-ae8e-3190c037d10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:20.279165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:20.311416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 548/1079] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:20.603351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf724-e936-4d60-9db2-2d4876712c84 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:20.604978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e4abf724-e936-4d60-9db2-2d4876712c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:20.677696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:20.770165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf724-e936-4d60-9db2-2d4876712c84 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:20.896647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321020a-2bb2-4610-95b7-d945708286e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:20.897169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321020a-2bb2-4610-95b7-d945708286e2 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.897442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321020a-2bb2-4610-95b7-d945708286e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.897707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321020a-2bb2-4610-95b7-d945708286e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.899976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5321020a-2bb2-4610-95b7-d945708286e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:20.919595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:20.999133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321020a-2bb2-4610-95b7-d945708286e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:21.000754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321020a-2bb2-4610-95b7-d945708286e2 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:21.021824 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 532/1080] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:23.336680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b2f4-689c-4673-9a3a-0cb38c7cf7c2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:23.337336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b2f4-689c-4673-9a3a-0cb38c7cf7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:23.337336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14b2f4-689c-4673-9a3a-0cb38c7cf7c2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:23.342284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 549/1081] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:23.344373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86d65-dc18-4781-9426-84c095ac101e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:23.344813 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86d65-dc18-4781-9426-84c095ac101e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:23.344933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86d65-dc18-4781-9426-84c095ac101e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:23.419676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fdf86d65-dc18-4781-9426-84c095ac101e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:23.420924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fdf86d65-dc18-4781-9426-84c095ac101e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:23.541427 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:23.550046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 533/1082] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:23.839666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14b2f4-689c-4673-9a3a-0cb38c7cf7c2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:23.841162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9f14b2f4-689c-4673-9a3a-0cb38c7cf7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:24.006767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14b2f4-689c-4673-9a3a-0cb38c7cf7c2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:24.070666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:24.138745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26953590-46e6-42d6-858d-a234ebd4c0a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:24.139736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953590-46e6-42d6-858d-a234ebd4c0a7 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:24.140099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953590-46e6-42d6-858d-a234ebd4c0a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:24.140419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953590-46e6-42d6-858d-a234ebd4c0a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:24.142192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-26953590-46e6-42d6-858d-a234ebd4c0a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:24.245749 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26953590-46e6-42d6-858d-a234ebd4c0a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:24.247520 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26953590-46e6-42d6-858d-a234ebd4c0a7 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:24.267255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 550/1083] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:26.553658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bbbcd-d3bb-4196-9aea-699371d43070 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.554288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bbbcd-d3bb-4196-9aea-699371d43070 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.554288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bbbcd-d3bb-4196-9aea-699371d43070 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.555789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 534/1084] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:26.560982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eb23e-6430-4e18-a170-acca7c51c6b3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.561425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eb23e-6430-4e18-a170-acca7c51c6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.561516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eb23e-6430-4e18-a170-acca7c51c6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.635168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-425eb23e-6430-4e18-a170-acca7c51c6b3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:26.636311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-425eb23e-6430-4e18-a170-acca7c51c6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:26.650531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:26.746130 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 551/1085] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:27.034191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bbbcd-d3bb-4196-9aea-699371d43070 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:27.035728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6d0bbbcd-d3bb-4196-9aea-699371d43070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.202256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bbbcd-d3bb-4196-9aea-699371d43070 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:27.281629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:27.334190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcc863-890b-49bb-b60a-df90c86fdc59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:27.334792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcc863-890b-49bb-b60a-df90c86fdc59 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.335066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcc863-890b-49bb-b60a-df90c86fdc59 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.335329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcc863-890b-49bb-b60a-df90c86fdc59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.337350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-11fcc863-890b-49bb-b60a-df90c86fdc59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.436137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcc863-890b-49bb-b60a-df90c86fdc59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:27.437828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcc863-890b-49bb-b60a-df90c86fdc59 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:27.457739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 535/1086] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:36.744633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=38 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:37.454286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=64 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:41.576155 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e3a-4e52-4f62-bbfb-9db0e6746fe7 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.576687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e3a-4e52-4f62-bbfb-9db0e6746fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.576806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e3a-4e52-4f62-bbfb-9db0e6746fe7 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.578141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 552/1087] 10.222.0.44 () {54 vars in 918 bytes} [Wed Oct 12 12:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:41.583094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73068c6-1189-487d-be87-e738c4e3fbe3 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.583485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73068c6-1189-487d-be87-e738c4e3fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.583689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73068c6-1189-487d-be87-e738c4e3fbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.657788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d73068c6-1189-487d-be87-e738c4e3fbe3 None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:41.658996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d73068c6-1189-487d-be87-e738c4e3fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:41.763262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 536/1088] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Oct 12 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:41.777044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b566a-60aa-4ce5-883b-7d3f5b6cd14d None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.777327 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b566a-60aa-4ce5-883b-7d3f5b6cd14d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.777626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b566a-60aa-4ce5-883b-7d3f5b6cd14d None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.779148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 553/1089] 10.222.0.44 () {54 vars in 918 bytes} [Wed Oct 12 12:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:41.783661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3483e-cb22-4ea8-8c96-d347d2ca6e70 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.783946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3483e-cb22-4ea8-8c96-d347d2ca6e70 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.784245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3483e-cb22-4ea8-8c96-d347d2ca6e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.856417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:41.859618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-24a3483e-cb22-4ea8-8c96-d347d2ca6e70 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:41.860803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-24a3483e-cb22-4ea8-8c96-d347d2ca6e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:41.975173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 537/1090] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Oct 12 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:44.527666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd3c75-6b24-4740-a601-e7cb0ce36511 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:44.528217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd3c75-6b24-4740-a601-e7cb0ce36511 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:44.528217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd3c75-6b24-4740-a601-e7cb0ce36511 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:44.529541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 554/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:44.534934 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c164650-ac57-497c-b89a-d9b54c8d267b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:44.535333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c164650-ac57-497c-b89a-d9b54c8d267b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:44.535517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c164650-ac57-497c-b89a-d9b54c8d267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:44.601052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8c164650-ac57-497c-b89a-d9b54c8d267b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:44.602261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8c164650-ac57-497c-b89a-d9b54c8d267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:44.656868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:44.699791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 538/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:44.989845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd3c75-6b24-4740-a601-e7cb0ce36511 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:44.991438 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-57fd3c75-6b24-4740-a601-e7cb0ce36511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:45.139261 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd3c75-6b24-4740-a601-e7cb0ce36511 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:45.211808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:45.250779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0deb3f6-0019-4377-8b22-d58ad4204df1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:45.251310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deb3f6-0019-4377-8b22-d58ad4204df1 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:45.251587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deb3f6-0019-4377-8b22-d58ad4204df1 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:45.251851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deb3f6-0019-4377-8b22-d58ad4204df1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:45.253422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d0deb3f6-0019-4377-8b22-d58ad4204df1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:45.342765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0deb3f6-0019-4377-8b22-d58ad4204df1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:45.344418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0deb3f6-0019-4377-8b22-d58ad4204df1 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:45.363183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 555/1093] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:46.502907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27101-d3c4-4ef0-908d-c16642bf61bb None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:46.503483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27101-d3c4-4ef0-908d-c16642bf61bb None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:46.503483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27101-d3c4-4ef0-908d-c16642bf61bb None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:46.504924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 539/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:46.511240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d9bd-fb6f-4130-a227-f78ca699c72e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:46.511671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d9bd-fb6f-4130-a227-f78ca699c72e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:46.511781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723d9bd-fb6f-4130-a227-f78ca699c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:46.586498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e723d9bd-fb6f-4130-a227-f78ca699c72e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:46.588259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e723d9bd-fb6f-4130-a227-f78ca699c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:46.615646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:46.683680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 556/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:46.832300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e27101-d3c4-4ef0-908d-c16642bf61bb None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:46.833814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f4e27101-d3c4-4ef0-908d-c16642bf61bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:46.976091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e27101-d3c4-4ef0-908d-c16642bf61bb None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:47.015924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:47.088358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e973af-5a9e-4a81-af43-377c64272ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:47.088964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e973af-5a9e-4a81-af43-377c64272ec3 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:47.089241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e973af-5a9e-4a81-af43-377c64272ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:47.089512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e973af-5a9e-4a81-af43-377c64272ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:47.091147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-54e973af-5a9e-4a81-af43-377c64272ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:47.180020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e973af-5a9e-4a81-af43-377c64272ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:47.181596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e973af-5a9e-4a81-af43-377c64272ec3 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:47.199915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 540/1096] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:48.328048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e739c3-1a51-4361-9cab-feada32ac270 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:48.328598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e739c3-1a51-4361-9cab-feada32ac270 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:48.328598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e739c3-1a51-4361-9cab-feada32ac270 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:48.329890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 557/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:48.336129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb054-6739-4a5d-8ab4-688dec21d0ef None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:48.336594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb054-6739-4a5d-8ab4-688dec21d0ef None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:48.336711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb054-6739-4a5d-8ab4-688dec21d0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:48.390795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:48.411889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-faddb054-6739-4a5d-8ab4-688dec21d0ef None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:48.413100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-faddb054-6739-4a5d-8ab4-688dec21d0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:48.511432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 541/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:48.659721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e739c3-1a51-4361-9cab-feada32ac270 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:48.661200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-90e739c3-1a51-4361-9cab-feada32ac270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:48.805691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e739c3-1a51-4361-9cab-feada32ac270 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:48.817489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:48.918336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2569-03f9-4471-9b66-c5e3aa67afcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:48.918934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2569-03f9-4471-9b66-c5e3aa67afcc service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:48.919207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2569-03f9-4471-9b66-c5e3aa67afcc service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:48.919472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2569-03f9-4471-9b66-c5e3aa67afcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:48.921034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b54b2569-03f9-4471-9b66-c5e3aa67afcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:49.008883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b2569-03f9-4471-9b66-c5e3aa67afcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:49.010493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b2569-03f9-4471-9b66-c5e3aa67afcc service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:49.028591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 558/1099] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:50.131290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec8baa-8664-4c5d-ae18-1fd297c98941 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:50.131798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec8baa-8664-4c5d-ae18-1fd297c98941 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:50.131895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec8baa-8664-4c5d-ae18-1fd297c98941 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:50.133184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 542/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:50.139344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f6067-a8d9-408e-a729-2b65e7a59429 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:50.139799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f6067-a8d9-408e-a729-2b65e7a59429 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:50.139896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f6067-a8d9-408e-a729-2b65e7a59429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:50.160190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:50.217392 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-350f6067-a8d9-408e-a729-2b65e7a59429 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:50.218623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-350f6067-a8d9-408e-a729-2b65e7a59429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:50.314906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 559/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:50.464068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec8baa-8664-4c5d-ae18-1fd297c98941 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:50.465561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2eec8baa-8664-4c5d-ae18-1fd297c98941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:50.574809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:50.611959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec8baa-8664-4c5d-ae18-1fd297c98941 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:50.732498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0952a7-f837-40cf-9455-7e2b70b3a53d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:50.733148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0952a7-f837-40cf-9455-7e2b70b3a53d service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:50.733429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0952a7-f837-40cf-9455-7e2b70b3a53d service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:50.733701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0952a7-f837-40cf-9455-7e2b70b3a53d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:50.735351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ee0952a7-f837-40cf-9455-7e2b70b3a53d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:50.818898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:50.833271 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0952a7-f837-40cf-9455-7e2b70b3a53d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:50.834963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0952a7-f837-40cf-9455-7e2b70b3a53d service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:50.854361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 543/1102] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:51.952698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cded8-5078-4ed3-8c5e-aa5d5763ab4a None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:51.953245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cded8-5078-4ed3-8c5e-aa5d5763ab4a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:51.953245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cded8-5078-4ed3-8c5e-aa5d5763ab4a None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:51.954661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 560/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:51.960612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c556f7-b8ae-439b-8fd1-7deff01221bc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:51.961065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c556f7-b8ae-439b-8fd1-7deff01221bc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:51.961210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c556f7-b8ae-439b-8fd1-7deff01221bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:52.035382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c6c556f7-b8ae-439b-8fd1-7deff01221bc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:52.036560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c6c556f7-b8ae-439b-8fd1-7deff01221bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:52.131968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 544/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:52.281668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cded8-5078-4ed3-8c5e-aa5d5763ab4a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:52.283139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ed1cded8-5078-4ed3-8c5e-aa5d5763ab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:52.369189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:52.430011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cded8-5078-4ed3-8c5e-aa5d5763ab4a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:52.542328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112878e-0cb3-437f-8926-a472ab41b111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:52.542961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112878e-0cb3-437f-8926-a472ab41b111 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:52.543234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112878e-0cb3-437f-8926-a472ab41b111 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:52.543501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112878e-0cb3-437f-8926-a472ab41b111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:52.545082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1112878e-0cb3-437f-8926-a472ab41b111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:52.593166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:52.633124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112878e-0cb3-437f-8926-a472ab41b111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:52.635249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112878e-0cb3-437f-8926-a472ab41b111 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:52.652716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 561/1105] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:53.751285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8462a-86eb-48ca-bc6c-a9b95788db3b None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:53.751851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8462a-86eb-48ca-bc6c-a9b95788db3b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:53.751851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8462a-86eb-48ca-bc6c-a9b95788db3b None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:53.753169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 545/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:53.759847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e478a-7c9d-4b14-852b-2dbc04378eed None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:53.760297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e478a-7c9d-4b14-852b-2dbc04378eed None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:53.760412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e478a-7c9d-4b14-852b-2dbc04378eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:53.835150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-985e478a-7c9d-4b14-852b-2dbc04378eed None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:53.836374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-985e478a-7c9d-4b14-852b-2dbc04378eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:53.931176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 562/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:54.080861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8462a-86eb-48ca-bc6c-a9b95788db3b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:54.082421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-edb8462a-86eb-48ca-bc6c-a9b95788db3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:54.131617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:54.227925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8462a-86eb-48ca-bc6c-a9b95788db3b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:54.338137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4dac6-a058-464c-84f1-6b7ea5425111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:54.338656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4dac6-a058-464c-84f1-6b7ea5425111 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:54.338929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4dac6-a058-464c-84f1-6b7ea5425111 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:54.339197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4dac6-a058-464c-84f1-6b7ea5425111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:54.340765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-73f4dac6-a058-464c-84f1-6b7ea5425111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:54.347675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:54.427739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4dac6-a058-464c-84f1-6b7ea5425111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:54.429310 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4dac6-a058-464c-84f1-6b7ea5425111 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:54.447344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 546/1108] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:55.545068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4f8d-c768-4bdb-82ae-24604696a3a4 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:55.545585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4f8d-c768-4bdb-82ae-24604696a3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:55.545585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4f8d-c768-4bdb-82ae-24604696a3a4 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:55.547017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 563/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:55.553136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7800d6-6ae9-49f5-8240-691d278c8a83 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:55.553577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7800d6-6ae9-49f5-8240-691d278c8a83 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:55.553669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7800d6-6ae9-49f5-8240-691d278c8a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:55.628346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0b7800d6-6ae9-49f5-8240-691d278c8a83 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:55.629556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0b7800d6-6ae9-49f5-8240-691d278c8a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:55.696002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:55.724236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 547/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:55.876656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4f8d-c768-4bdb-82ae-24604696a3a4 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:55.878128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-760a4f8d-c768-4bdb-82ae-24604696a3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:55.895466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:56.022312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4f8d-c768-4bdb-82ae-24604696a3a4 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:56.109068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:56.134271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd991b-d49b-4b38-a511-c2d29963229c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:56.134816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd991b-d49b-4b38-a511-c2d29963229c service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:56.135088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd991b-d49b-4b38-a511-c2d29963229c service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:56.135353 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd991b-d49b-4b38-a511-c2d29963229c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:56.136960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-23cd991b-d49b-4b38-a511-c2d29963229c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:56.226119 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd991b-d49b-4b38-a511-c2d29963229c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:56.227754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd991b-d49b-4b38-a511-c2d29963229c service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:56.246146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 564/1111] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:57.333833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264e3a7-05f3-41f0-8a9a-0ba94565383d None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:57.334402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264e3a7-05f3-41f0-8a9a-0ba94565383d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:57.334402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264e3a7-05f3-41f0-8a9a-0ba94565383d None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:57.335659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 548/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:57.342683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdda0c3-b476-4f33-8949-1d50b28d8a47 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:57.343103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdda0c3-b476-4f33-8949-1d50b28d8a47 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:57.343230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdda0c3-b476-4f33-8949-1d50b28d8a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:57.417961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-bcdda0c3-b476-4f33-8949-1d50b28d8a47 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:57.419170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bcdda0c3-b476-4f33-8949-1d50b28d8a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:57.460182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:57.516538 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 565/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:57.666530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264e3a7-05f3-41f0-8a9a-0ba94565383d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:57.668059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6264e3a7-05f3-41f0-8a9a-0ba94565383d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:57.812458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264e3a7-05f3-41f0-8a9a-0ba94565383d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:57.867327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:57.925516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44ff7e-679f-4560-b67f-3457e0a7a94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:57.926144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44ff7e-679f-4560-b67f-3457e0a7a94b service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:57.926521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44ff7e-679f-4560-b67f-3457e0a7a94b service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:57.926798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44ff7e-679f-4560-b67f-3457e0a7a94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:57.928387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0a44ff7e-679f-4560-b67f-3457e0a7a94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:58.017236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44ff7e-679f-4560-b67f-3457e0a7a94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:58.018845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44ff7e-679f-4560-b67f-3457e0a7a94b service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:58.036341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 549/1114] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:59.137663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aab60-40b1-4bc0-bf97-f11be237ddd2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:59.138251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aab60-40b1-4bc0-bf97-f11be237ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:59.138251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aab60-40b1-4bc0-bf97-f11be237ddd2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:59.139581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 566/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:59.145825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ca3c4-b698-4f71-9514-32a02ca8dbe7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:59.146295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ca3c4-b698-4f71-9514-32a02ca8dbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:59.146387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ca3c4-b698-4f71-9514-32a02ca8dbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:59.220540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c42ca3c4-b698-4f71-9514-32a02ca8dbe7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:59.221730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c42ca3c4-b698-4f71-9514-32a02ca8dbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:59.223714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:59.316348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 550/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:59.465341 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aab60-40b1-4bc0-bf97-f11be237ddd2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:59.466913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4e5aab60-40b1-4bc0-bf97-f11be237ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:59.611662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aab60-40b1-4bc0-bf97-f11be237ddd2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:59.638662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:59.724123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58609b9-9585-4282-baab-c10697c6bf2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:59.724728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58609b9-9585-4282-baab-c10697c6bf2a service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:59.725004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58609b9-9585-4282-baab-c10697c6bf2a service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:59.725269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58609b9-9585-4282-baab-c10697c6bf2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:59.726934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e58609b9-9585-4282-baab-c10697c6bf2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:59.816587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58609b9-9585-4282-baab-c10697c6bf2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:59.818189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58609b9-9585-4282-baab-c10697c6bf2a service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:59.835687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 567/1117] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:01.889060 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4947-40cb-493e-af0d-3cc9d4f806ed None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:01.889652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4947-40cb-493e-af0d-3cc9d4f806ed None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:01.889652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4947-40cb-493e-af0d-3cc9d4f806ed None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:01.891005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 551/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:01.896335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0be30-8828-4f3e-a49f-56bf0583727f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:01.896756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0be30-8828-4f3e-a49f-56bf0583727f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:01.896871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0be30-8828-4f3e-a49f-56bf0583727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:01.937951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:01.962481 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d5f0be30-8828-4f3e-a49f-56bf0583727f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:01.963675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d5f0be30-8828-4f3e-a49f-56bf0583727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:02.060370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 568/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:02.345671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4947-40cb-493e-af0d-3cc9d4f806ed None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.347272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-45ea4947-40cb-493e-af0d-3cc9d4f806ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:02.448493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4947-40cb-493e-af0d-3cc9d4f806ed None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.482587 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:02.524550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bc23c-06f9-4fff-a09b-4fae1d17ff96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.525176 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc23c-06f9-4fff-a09b-4fae1d17ff96 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.525466 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc23c-06f9-4fff-a09b-4fae1d17ff96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.525735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bc23c-06f9-4fff-a09b-4fae1d17ff96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.527354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2b0bc23c-06f9-4fff-a09b-4fae1d17ff96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:02.635232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bc23c-06f9-4fff-a09b-4fae1d17ff96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.636569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bc23c-06f9-4fff-a09b-4fae1d17ff96 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.654166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 552/1120] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:04.929551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a0697-0435-464b-8261-549dc31788f2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:04.930154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a0697-0435-464b-8261-549dc31788f2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:04.930154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a0697-0435-464b-8261-549dc31788f2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:04.932158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 569/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:04.937314 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5783e-dc92-4a18-8cea-2015f26533ba None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:04.937744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5783e-dc92-4a18-8cea-2015f26533ba None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:04.937854 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5783e-dc92-4a18-8cea-2015f26533ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:05.003145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1ee5783e-dc92-4a18-8cea-2015f26533ba None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:05.004341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1ee5783e-dc92-4a18-8cea-2015f26533ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:05.016943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:05.103260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 553/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:05.385971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a0697-0435-464b-8261-549dc31788f2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:05.387511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c84a0697-0435-464b-8261-549dc31788f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:05.489297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a0697-0435-464b-8261-549dc31788f2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:05.501912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:05.562815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe0b75-4eaa-4f04-be80-cdfb49aa87d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:05.563416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0b75-4eaa-4f04-be80-cdfb49aa87d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:05.563696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0b75-4eaa-4f04-be80-cdfb49aa87d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:05.563972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0b75-4eaa-4f04-be80-cdfb49aa87d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:05.565544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2dbe0b75-4eaa-4f04-be80-cdfb49aa87d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:05.672982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe0b75-4eaa-4f04-be80-cdfb49aa87d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:05.674354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe0b75-4eaa-4f04-be80-cdfb49aa87d8 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:05.691873 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 570/1123] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:08.165144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdb70a-17f3-4d40-8d24-c18bb96eeddf None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.165796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdb70a-17f3-4d40-8d24-c18bb96eeddf None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.165796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdb70a-17f3-4d40-8d24-c18bb96eeddf None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.167120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 554/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:08.172290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfe1eb-1ad6-4067-8f8b-6a7fb35a79ab None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.172675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfe1eb-1ad6-4067-8f8b-6a7fb35a79ab None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.172838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfe1eb-1ad6-4067-8f8b-6a7fb35a79ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.220660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:08.238267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-20cfe1eb-1ad6-4067-8f8b-6a7fb35a79ab None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:08.239480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-20cfe1eb-1ad6-4067-8f8b-6a7fb35a79ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:08.336571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 571/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:08.624511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdb70a-17f3-4d40-8d24-c18bb96eeddf None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:08.626152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-94fdb70a-17f3-4d40-8d24-c18bb96eeddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:08.730019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdb70a-17f3-4d40-8d24-c18bb96eeddf None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:08.770316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:08.804017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa00bf-0f07-4346-92d6-fb2d3dbd5ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:08.804669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa00bf-0f07-4346-92d6-fb2d3dbd5ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.804943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa00bf-0f07-4346-92d6-fb2d3dbd5ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.805206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa00bf-0f07-4346-92d6-fb2d3dbd5ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.807399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6baa00bf-0f07-4346-92d6-fb2d3dbd5ef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:08.912946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa00bf-0f07-4346-92d6-fb2d3dbd5ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:08.914320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa00bf-0f07-4346-92d6-fb2d3dbd5ef1 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:08.933338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 555/1126] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:11.186864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6e269-8d65-4966-a094-3759f2d74736 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.187370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6e269-8d65-4966-a094-3759f2d74736 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.187488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6e269-8d65-4966-a094-3759f2d74736 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.188840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 572/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:11.194323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c6cb0-cce3-4258-b70a-356edaac9324 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.195381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c6cb0-cce3-4258-b70a-356edaac9324 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.195381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c6cb0-cce3-4258-b70a-356edaac9324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.263013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c36c6cb0-cce3-4258-b70a-356edaac9324 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:11.264222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c36c6cb0-cce3-4258-b70a-356edaac9324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.284523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:11.359008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 556/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:11.639853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6e269-8d65-4966-a094-3759f2d74736 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.641379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-74d6e269-8d65-4966-a094-3759f2d74736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.743000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6e269-8d65-4966-a094-3759f2d74736 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.762923 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:11.816549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e667a1-92f5-4120-a10d-c8452db6909b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.817149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e667a1-92f5-4120-a10d-c8452db6909b service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.817422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e667a1-92f5-4120-a10d-c8452db6909b service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.817691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e667a1-92f5-4120-a10d-c8452db6909b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.819292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-77e667a1-92f5-4120-a10d-c8452db6909b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.925137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e667a1-92f5-4120-a10d-c8452db6909b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.926494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e667a1-92f5-4120-a10d-c8452db6909b service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.945026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 573/1129] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:14.151574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29995640-d70f-4a19-8a4b-5b751bd07eea None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.156413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29995640-d70f-4a19-8a4b-5b751bd07eea None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.156413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29995640-d70f-4a19-8a4b-5b751bd07eea None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.156413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 557/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:32:14.158860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2cf3-3791-4f79-bda2-73edd70b506e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.159306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2cf3-3791-4f79-bda2-73edd70b506e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.159415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2cf3-3791-4f79-bda2-73edd70b506e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.224633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:14.228193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-0d6f2cf3-3791-4f79-bda2-73edd70b506e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:14.229394 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0d6f2cf3-3791-4f79-bda2-73edd70b506e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.329809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 574/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:14.612866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29995640-d70f-4a19-8a4b-5b751bd07eea None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.614416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-29995640-d70f-4a19-8a4b-5b751bd07eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.724766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29995640-d70f-4a19-8a4b-5b751bd07eea None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.778387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:14.802328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08046489-dee8-41f4-9386-4683c2873d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.802964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08046489-dee8-41f4-9386-4683c2873d31 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.803244 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08046489-dee8-41f4-9386-4683c2873d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.803510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08046489-dee8-41f4-9386-4683c2873d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.805109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-08046489-dee8-41f4-9386-4683c2873d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.920788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08046489-dee8-41f4-9386-4683c2873d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.922316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08046489-dee8-41f4-9386-4683c2873d31 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.940337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 558/1132] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:17.188372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0c6c-5750-4073-a09a-1fc2652ef33b None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.188916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0c6c-5750-4073-a09a-1fc2652ef33b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.188916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0c6c-5750-4073-a09a-1fc2652ef33b None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.190616 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 575/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:17.195948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ea32d-5122-4d25-887b-09948f0f0258 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.196396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ea32d-5122-4d25-887b-09948f0f0258 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.202022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ea32d-5122-4d25-887b-09948f0f0258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.266030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a98ea32d-5122-4d25-887b-09948f0f0258 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:17.267359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a98ea32d-5122-4d25-887b-09948f0f0258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.298049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:17.368430 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 559/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:17.654712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c0c6c-5750-4073-a09a-1fc2652ef33b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.656252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e16c0c6c-5750-4073-a09a-1fc2652ef33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.772891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c0c6c-5750-4073-a09a-1fc2652ef33b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.803014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:17.850459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb8aba-9a99-4c2c-a201-5e937f422449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.851068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8aba-9a99-4c2c-a201-5e937f422449 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.851344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8aba-9a99-4c2c-a201-5e937f422449 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.851611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb8aba-9a99-4c2c-a201-5e937f422449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.853739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b7fb8aba-9a99-4c2c-a201-5e937f422449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.969741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb8aba-9a99-4c2c-a201-5e937f422449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.971122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb8aba-9a99-4c2c-a201-5e937f422449 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.989861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 576/1135] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:20.207103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17c22-6f34-40a0-98e3-067e6f3a998a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.207668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17c22-6f34-40a0-98e3-067e6f3a998a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.207668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17c22-6f34-40a0-98e3-067e6f3a998a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.209002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 560/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:20.214360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8fc99-8706-45b6-817b-9095669b9029 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.214867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8fc99-8706-45b6-817b-9095669b9029 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.214983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8fc99-8706-45b6-817b-9095669b9029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.280781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-45d8fc99-8706-45b6-817b-9095669b9029 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:20.282007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-45d8fc99-8706-45b6-817b-9095669b9029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:20.290321 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:20.380687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 577/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:20.661758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde17c22-6f34-40a0-98e3-067e6f3a998a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.663328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bde17c22-6f34-40a0-98e3-067e6f3a998a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:20.772317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde17c22-6f34-40a0-98e3-067e6f3a998a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.828612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:20.846764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11881e-ff0d-41a4-813f-f9b3c2ba1358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.847377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11881e-ff0d-41a4-813f-f9b3c2ba1358 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.847656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11881e-ff0d-41a4-813f-f9b3c2ba1358 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.847928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11881e-ff0d-41a4-813f-f9b3c2ba1358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.849521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-af11881e-ff0d-41a4-813f-f9b3c2ba1358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:20.959155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11881e-ff0d-41a4-813f-f9b3c2ba1358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.960620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11881e-ff0d-41a4-813f-f9b3c2ba1358 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.979108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 561/1138] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:23.190630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419139c2-7d31-4cdc-9312-bfafc9138560 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:23.191185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419139c2-7d31-4cdc-9312-bfafc9138560 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:23.191185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419139c2-7d31-4cdc-9312-bfafc9138560 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:23.192529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 578/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:23.197850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00a7d-7ecf-438b-847e-fb1210120844 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:23.198399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00a7d-7ecf-438b-847e-fb1210120844 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:23.198510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00a7d-7ecf-438b-847e-fb1210120844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:23.264129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-87e00a7d-7ecf-438b-847e-fb1210120844 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:23.265321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-87e00a7d-7ecf-438b-847e-fb1210120844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:23.303660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:23.361165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 562/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:23.641352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419139c2-7d31-4cdc-9312-bfafc9138560 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:23.642903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-419139c2-7d31-4cdc-9312-bfafc9138560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:23.744832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419139c2-7d31-4cdc-9312-bfafc9138560 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:23.782888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:23.817607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146d9ea-9daa-4fae-8db4-b5b6b3f8d06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:23.818239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d9ea-9daa-4fae-8db4-b5b6b3f8d06d service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:23.818518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d9ea-9daa-4fae-8db4-b5b6b3f8d06d service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:23.818790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146d9ea-9daa-4fae-8db4-b5b6b3f8d06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:23.820370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6146d9ea-9daa-4fae-8db4-b5b6b3f8d06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:23.927989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146d9ea-9daa-4fae-8db4-b5b6b3f8d06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:23.929373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146d9ea-9daa-4fae-8db4-b5b6b3f8d06d service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:23.947951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 579/1141] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:26.093998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e318c-6742-4970-90dc-62576fcc413e None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.094490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e318c-6742-4970-90dc-62576fcc413e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.094628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e318c-6742-4970-90dc-62576fcc413e None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:26.096053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 563/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:26.101443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eecfa9-ee7f-4783-b297-12a77e71667c None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.101858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eecfa9-ee7f-4783-b297-12a77e71667c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.102005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eecfa9-ee7f-4783-b297-12a77e71667c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:26.167563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-82eecfa9-ee7f-4783-b297-12a77e71667c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:26.168791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-82eecfa9-ee7f-4783-b297-12a77e71667c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.186620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:26.265807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 580/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:26.550268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e318c-6742-4970-90dc-62576fcc413e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:26.551811 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9c9e318c-6742-4970-90dc-62576fcc413e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.655102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e318c-6742-4970-90dc-62576fcc413e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:26.721338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:26.728298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6720f9-38e9-4677-b938-4070c5e5db28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:26.728902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6720f9-38e9-4677-b938-4070c5e5db28 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.729179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6720f9-38e9-4677-b938-4070c5e5db28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.729450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6720f9-38e9-4677-b938-4070c5e5db28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:26.731077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0a6720f9-38e9-4677-b938-4070c5e5db28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.841634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6720f9-38e9-4677-b938-4070c5e5db28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:26.843118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6720f9-38e9-4677-b938-4070c5e5db28 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:26.861413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 564/1144] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:29.003752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7a1f-bc3a-4784-8cda-fedb4cb476ab None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.004325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7a1f-bc3a-4784-8cda-fedb4cb476ab None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.004325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7a1f-bc3a-4784-8cda-fedb4cb476ab None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.005667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 581/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:29.011735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f366b1-0db6-4ae7-9a7b-72fecee0dce6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.012156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f366b1-0db6-4ae7-9a7b-72fecee0dce6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.012309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f366b1-0db6-4ae7-9a7b-72fecee0dce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.077860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f6f366b1-0db6-4ae7-9a7b-72fecee0dce6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:29.079087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f6f366b1-0db6-4ae7-9a7b-72fecee0dce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:29.126194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:29.173817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 565/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:29.457308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba7a1f-bc3a-4784-8cda-fedb4cb476ab None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.458910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-36ba7a1f-bc3a-4784-8cda-fedb4cb476ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:29.560536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba7a1f-bc3a-4784-8cda-fedb4cb476ab None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.608171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:29.633264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243453d2-f80c-4ef3-9547-b39d2de84996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.633880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243453d2-f80c-4ef3-9547-b39d2de84996 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.634201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243453d2-f80c-4ef3-9547-b39d2de84996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.634506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243453d2-f80c-4ef3-9547-b39d2de84996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.636081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-243453d2-f80c-4ef3-9547-b39d2de84996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:29.748976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243453d2-f80c-4ef3-9547-b39d2de84996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.750365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243453d2-f80c-4ef3-9547-b39d2de84996 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.768755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 582/1147] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:39.172332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=22 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:39.766390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=66 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:49.672557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba5986-b982-4ef6-a802-fdda8b933526 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:49.673097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba5986-b982-4ef6-a802-fdda8b933526 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:49.673097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba5986-b982-4ef6-a802-fdda8b933526 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:49.674533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 566/1148] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:32:49.679845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4457f-5e70-4b50-a3c0-65154c61b937 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:49.680278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4457f-5e70-4b50-a3c0-65154c61b937 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:49.680383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4457f-5e70-4b50-a3c0-65154c61b937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:49.746701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7ce4457f-5e70-4b50-a3c0-65154c61b937 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:49.747901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7ce4457f-5e70-4b50-a3c0-65154c61b937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:49.842825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 583/1149] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:50.139533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f34b7-08bd-4910-b3c4-9ca840ddcafb None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.139815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f34b7-08bd-4910-b3c4-9ca840ddcafb None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.140089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f34b7-08bd-4910-b3c4-9ca840ddcafb None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.141415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 567/1150] 10.222.0.44 () {54 vars in 962 bytes} [Wed Oct 12 12:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:50.146531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39994367-44ad-4143-abcc-d69c4083ee0e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.146816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39994367-44ad-4143-abcc-d69c4083ee0e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.147089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39994367-44ad-4143-abcc-d69c4083ee0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.223676 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-39994367-44ad-4143-abcc-d69c4083ee0e None None] MFA Rules not processed for user `70cbcaeb843e470781661f00ec91bffd`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:50.224869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-39994367-44ad-4143-abcc-d69c4083ee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:50.245678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:50.300832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 584/1151] 10.222.0.44 () {58 vars in 1059 bytes} [Wed Oct 12 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:50.309155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f34b7-08bd-4910-b3c4-9ca840ddcafb None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:50.310706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e38f34b7-08bd-4910-b3c4-9ca840ddcafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:50.415150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f34b7-08bd-4910-b3c4-9ca840ddcafb None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.488071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093e299-d4b2-4e98-bf13-e42ba50de523 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cbcaeb843e470781661f00ec91bffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.488667 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e299-d4b2-4e98-bf13-e42ba50de523 service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.488946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e299-d4b2-4e98-bf13-e42ba50de523 service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.489219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e299-d4b2-4e98-bf13-e42ba50de523 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.490844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d093e299-d4b2-4e98-bf13-e42ba50de523 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:50.544167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:50.598960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093e299-d4b2-4e98-bf13-e42ba50de523 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.600365 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093e299-d4b2-4e98-bf13-e42ba50de523 service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.624768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 568/1152] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:52.842808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba2e9d-03ce-4230-ba96-cce3d278f0f2 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:52.843395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba2e9d-03ce-4230-ba96-cce3d278f0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:52.843395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba2e9d-03ce-4230-ba96-cce3d278f0f2 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:52.844709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 585/1153] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:52.849927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e212a2-a7ed-40d4-b1c6-097fa9fd3b67 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:52.850384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e212a2-a7ed-40d4-b1c6-097fa9fd3b67 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:52.850495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e212a2-a7ed-40d4-b1c6-097fa9fd3b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:52.914855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-13e212a2-a7ed-40d4-b1c6-097fa9fd3b67 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:52.916582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-13e212a2-a7ed-40d4-b1c6-097fa9fd3b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:53.012082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 569/1154] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:53.286880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba2e9d-03ce-4230-ba96-cce3d278f0f2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.288389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4cba2e9d-03ce-4230-ba96-cce3d278f0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:53.412360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba2e9d-03ce-4230-ba96-cce3d278f0f2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.461146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:53.506591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b97e4-b0bc-47bd-b1a3-037ba6a9386e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.507190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b97e4-b0bc-47bd-b1a3-037ba6a9386e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.507473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b97e4-b0bc-47bd-b1a3-037ba6a9386e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.507746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b97e4-b0bc-47bd-b1a3-037ba6a9386e admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.508382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b97e4-b0bc-47bd-b1a3-037ba6a9386e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.510124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-2f8b97e4-b0bc-47bd-b1a3-037ba6a9386e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:32:53.510124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:32:53.510124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:32:53.511327 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b97e4-b0bc-47bd-b1a3-037ba6a9386e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.530082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 586/1155] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Oct 12 12:32:53 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:55.676368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088cb1-a5bf-46eb-9ca0-f71c5635e836 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:55.676908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088cb1-a5bf-46eb-9ca0-f71c5635e836 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:55.676908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088cb1-a5bf-46eb-9ca0-f71c5635e836 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:55.678298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 570/1156] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:55.683574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0f7d8-108a-41e5-93d9-56756e336fa0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:55.683985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0f7d8-108a-41e5-93d9-56756e336fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:55.684147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0f7d8-108a-41e5-93d9-56756e336fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:55.749381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-04b0f7d8-108a-41e5-93d9-56756e336fa0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:55.750593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-04b0f7d8-108a-41e5-93d9-56756e336fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:55.845222 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 587/1157] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:56.139559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88088cb1-a5bf-46eb-9ca0-f71c5635e836 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:56.141133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-88088cb1-a5bf-46eb-9ca0-f71c5635e836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:56.151874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:56.286796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88088cb1-a5bf-46eb-9ca0-f71c5635e836 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:56.366694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:56.399692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad92082-7d9d-4f00-848e-1779b666b348 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:56.406621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad92082-7d9d-4f00-848e-1779b666b348 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:56.406621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad92082-7d9d-4f00-848e-1779b666b348 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:56.406621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad92082-7d9d-4f00-848e-1779b666b348 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:56.406621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6ad92082-7d9d-4f00-848e-1779b666b348 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:56.493892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad92082-7d9d-4f00-848e-1779b666b348 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:56.495494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad92082-7d9d-4f00-848e-1779b666b348 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:56.513747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 571/1158] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:58.819356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3d81-2ea2-4765-b2d9-412782b1a523 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:58.819905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3d81-2ea2-4765-b2d9-412782b1a523 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:58.819905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3d81-2ea2-4765-b2d9-412782b1a523 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:58.821302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 588/1159] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:32:58.827139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c9dd7-6e38-43ab-97e5-acda14840500 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:58.827611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c9dd7-6e38-43ab-97e5-acda14840500 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:58.827713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c9dd7-6e38-43ab-97e5-acda14840500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:58.893008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b40c9dd7-6e38-43ab-97e5-acda14840500 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:58.894223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b40c9dd7-6e38-43ab-97e5-acda14840500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:58.939398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:58.989150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 572/1160] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:59.277958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb3d81-2ea2-4765-b2d9-412782b1a523 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:59.279596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4abb3d81-2ea2-4765-b2d9-412782b1a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:59.304780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:59.384831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb3d81-2ea2-4765-b2d9-412782b1a523 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:59.456345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f80e23-571e-4e3d-ab50-cd80c612ec6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:59.456941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f80e23-571e-4e3d-ab50-cd80c612ec6b service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:59.457220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f80e23-571e-4e3d-ab50-cd80c612ec6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:59.457493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f80e23-571e-4e3d-ab50-cd80c612ec6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:59.459112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e8f80e23-571e-4e3d-ab50-cd80c612ec6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:59.533245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:59.568509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f80e23-571e-4e3d-ab50-cd80c612ec6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:59.570001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f80e23-571e-4e3d-ab50-cd80c612ec6b service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:59.588601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 589/1161] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:01.770456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab37c4-7267-475c-9dec-15f4e44e8164 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:01.771026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab37c4-7267-475c-9dec-15f4e44e8164 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:01.771026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab37c4-7267-475c-9dec-15f4e44e8164 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:01.772343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 573/1162] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:33:01.778182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbcaf6-f262-4159-8a54-7763e4c50976 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:01.778626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbcaf6-f262-4159-8a54-7763e4c50976 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:01.778757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbcaf6-f262-4159-8a54-7763e4c50976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:01.844006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-36dbcaf6-f262-4159-8a54-7763e4c50976 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:01.845190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-36dbcaf6-f262-4159-8a54-7763e4c50976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:01.939673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 590/1163] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:02.230127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab37c4-7267-475c-9dec-15f4e44e8164 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:02.231680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4fab37c4-7267-475c-9dec-15f4e44e8164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:02.332631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab37c4-7267-475c-9dec-15f4e44e8164 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:02.403682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:02.405316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d8bd2-2b56-45c4-b02e-1f7e381b6c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:02.405905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d8bd2-2b56-45c4-b02e-1f7e381b6c41 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:02.406205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d8bd2-2b56-45c4-b02e-1f7e381b6c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:02.406474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d8bd2-2b56-45c4-b02e-1f7e381b6c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:02.408049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-736d8bd2-2b56-45c4-b02e-1f7e381b6c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:02.518136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d8bd2-2b56-45c4-b02e-1f7e381b6c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:02.519588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d8bd2-2b56-45c4-b02e-1f7e381b6c41 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:02.538332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 574/1164] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:04.764889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6e1ac-a6a3-4e7c-96d4-ab7f74d00ea5 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:04.765402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6e1ac-a6a3-4e7c-96d4-ab7f74d00ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:04.765402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6e1ac-a6a3-4e7c-96d4-ab7f74d00ea5 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:04.766725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 591/1165] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:33:04.771875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d726-c856-4a95-8669-6b1471bd9a7e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:04.772329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d726-c856-4a95-8669-6b1471bd9a7e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:04.772433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1d726-c856-4a95-8669-6b1471bd9a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:04.837775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d1a1d726-c856-4a95-8669-6b1471bd9a7e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:04.839013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d1a1d726-c856-4a95-8669-6b1471bd9a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:04.891969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:04.935842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 575/1166] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:05.234992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6e1ac-a6a3-4e7c-96d4-ab7f74d00ea5 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:05.236543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-76a6e1ac-a6a3-4e7c-96d4-ab7f74d00ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:05.271612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:05.340036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6e1ac-a6a3-4e7c-96d4-ab7f74d00ea5 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:05.411729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad76f5-d039-4650-9e56-b7e7c998aa78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:05.412320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad76f5-d039-4650-9e56-b7e7c998aa78 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:05.412598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad76f5-d039-4650-9e56-b7e7c998aa78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:05.412864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad76f5-d039-4650-9e56-b7e7c998aa78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:05.414475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f9ad76f5-d039-4650-9e56-b7e7c998aa78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:05.493625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:05.521263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad76f5-d039-4650-9e56-b7e7c998aa78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:05.522693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad76f5-d039-4650-9e56-b7e7c998aa78 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:05.541669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 592/1167] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:08.071642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4b21-f25b-4472-b02a-d262473322f6 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:08.072073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4b21-f25b-4472-b02a-d262473322f6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:08.072201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a4b21-f25b-4472-b02a-d262473322f6 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:08.073567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 576/1168] 10.222.0.44 () {54 vars in 927 bytes} [Wed Oct 12 12:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:33:08.079052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06fb8d-4185-4241-bcfe-33a97f826f77 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:08.079490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06fb8d-4185-4241-bcfe-33a97f826f77 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:08.079604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06fb8d-4185-4241-bcfe-33a97f826f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:08.145837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9d06fb8d-4185-4241-bcfe-33a97f826f77 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:08.147072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9d06fb8d-4185-4241-bcfe-33a97f826f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:08.244842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 593/1169] 10.222.0.44 () {58 vars in 1024 bytes} [Wed Oct 12 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:08.533923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a4b21-f25b-4472-b02a-d262473322f6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:08.535480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-101a4b21-f25b-4472-b02a-d262473322f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:08.638335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a4b21-f25b-4472-b02a-d262473322f6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:08.710964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb9878-a094-48ca-81a8-0cc94b4a185b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:08.711593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb9878-a094-48ca-81a8-0cc94b4a185b service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:08.711877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb9878-a094-48ca-81a8-0cc94b4a185b service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:08.712149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb9878-a094-48ca-81a8-0cc94b4a185b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:08.713738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e8cb9878-a094-48ca-81a8-0cc94b4a185b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:08.722191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:08.823793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb9878-a094-48ca-81a8-0cc94b4a185b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:08.825247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb9878-a094-48ca-81a8-0cc94b4a185b service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:08.843197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 577/1170] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:18.242805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=82 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:18.840748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=50 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:29.845805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e4ff-ba23-42b0-830b-5d740d9022ba None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:29.846428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e4ff-ba23-42b0-830b-5d740d9022ba None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:29.846428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e4ff-ba23-42b0-830b-5d740d9022ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:29.921852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-0345e4ff-ba23-42b0-830b-5d740d9022ba None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:29.923064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0345e4ff-ba23-42b0-830b-5d740d9022ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.016703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 594/1171] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:30.024681 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a6507-5a83-47eb-a3a0-677d0fff7e88 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.025184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a6507-5a83-47eb-a3a0-677d0fff7e88 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.025301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a6507-5a83-47eb-a3a0-677d0fff7e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.101124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d94a6507-5a83-47eb-a3a0-677d0fff7e88 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:30.102315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d94a6507-5a83-47eb-a3a0-677d0fff7e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.197351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 578/1172] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:30.202262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345e4ff-ba23-42b0-830b-5d740d9022ba None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.203459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0345e4ff-ba23-42b0-830b-5d740d9022ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.215303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345e4ff-ba23-42b0-830b-5d740d9022ba None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.275010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:30.307834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.308436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.308714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.308982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.309615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:34:30.309914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.311089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:34:30.311089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:34:30.311089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:34:30.311974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.324061 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 595/1173] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:34:30.333603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a6507-5a83-47eb-a3a0-677d0fff7e88 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.343621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a6507-5a83-47eb-a3a0-677d0fff7e88 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.404969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:30.438009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.438623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.438900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.439171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.439800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.440900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:34:30.440900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 12 12:34:30.440900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: Oct 12 12:34:30.442119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.531421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 579/1174] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:34:30 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:30.535620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.537124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.569134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86dd89-d7c6-4115-b503-43d2b025f58b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.663180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.663698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.663971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.664238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.664860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.665857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:34:30.665857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:34:30.665857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:34:30.667096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.781631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 596/1175] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:34:30 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:30.785807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.787352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.848877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242d594-27f8-4ee1-9a73-efc7c57cddce admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.932104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:30.942767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.944170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.944170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.944170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.944499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.946083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.959329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 580/1176] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:30.963352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.964863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.976169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34094c3-78af-4468-8b79-7cc6f2c2185c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.001305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:31.071071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125d396-a07c-447d-bb74-a5ea322d3e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.071683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d396-a07c-447d-bb74-a5ea322d3e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.071965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d396-a07c-447d-bb74-a5ea322d3e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.072234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125d396-a07c-447d-bb74-a5ea322d3e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.072903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125d396-a07c-447d-bb74-a5ea322d3e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.074669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125d396-a07c-447d-bb74-a5ea322d3e26 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.087745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 597/1177] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:31.092053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.093599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:31.127188 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f519d-ea95-4e04-8021-bf7b82389dbe admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.219519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa090c1c-c0a8-433b-b6ec-18c5a85ae9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.220094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090c1c-c0a8-433b-b6ec-18c5a85ae9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.220366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090c1c-c0a8-433b-b6ec-18c5a85ae9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.220631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090c1c-c0a8-433b-b6ec-18c5a85ae9d7 admin admin] PATH_INFO: `/v3/projects/2f71b87e82764dd7ad04da030d1e514e/users/077d9c2cd616436bb37ee8a9b7e4a336/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.243933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa090c1c-c0a8-433b-b6ec-18c5a85ae9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f71b87e82764dd7ad04da030d1e514e, user_id=077d9c2cd616436bb37ee8a9b7e4a336, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.247290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa090c1c-c0a8-433b-b6ec-18c5a85ae9d7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.334076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 581/1178] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:31 2022] PUT /identity/v3/projects/2f71b87e82764dd7ad04da030d1e514e/users/077d9c2cd616436bb37ee8a9b7e4a336/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:31.345127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9ec1-ac66-4115-958a-db7e6fa98b71 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.345452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9ec1-ac66-4115-958a-db7e6fa98b71 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.345729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9ec1-ac66-4115-958a-db7e6fa98b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.403566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ea6b9ec1-ac66-4115-958a-db7e6fa98b71 None None] MFA Rules not processed for user `077d9c2cd616436bb37ee8a9b7e4a336`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:31.405063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ea6b9ec1-ac66-4115-958a-db7e6fa98b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:31.481056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 598/1179] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:31.494177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7fa5a-8881-45cf-b051-d83217b240f2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.494460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7fa5a-8881-45cf-b051-d83217b240f2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.494728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7fa5a-8881-45cf-b051-d83217b240f2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.496095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 582/1180] 10.222.0.44 () {54 vars in 963 bytes} [Wed Oct 12 12:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:34:31.500816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7faa40-0e3d-40b3-9e2b-e4ca8a4c4bf0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.501131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7faa40-0e3d-40b3-9e2b-e4ca8a4c4bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.501404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7faa40-0e3d-40b3-9e2b-e4ca8a4c4bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.532456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:31.574978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ba7faa40-0e3d-40b3-9e2b-e4ca8a4c4bf0 None None] MFA Rules not processed for user `8238fbc3012844ea8b2b5ab13c8ea5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:31.576158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ba7faa40-0e3d-40b3-9e2b-e4ca8a4c4bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:31.650283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 599/1181] 10.222.0.44 () {58 vars in 1060 bytes} [Wed Oct 12 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:31.658719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7fa5a-8881-45cf-b051-d83217b240f2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.660237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c3d7fa5a-8881-45cf-b051-d83217b240f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:31.709962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:31.763665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7fa5a-8881-45cf-b051-d83217b240f2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.838520 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.839715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.839999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.840267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.841848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:31.938100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.939502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.942935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:31.957545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 583/1182] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:34:32.002369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b609376-a1f5-450d-acfb-481d3cc5a4c5 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:32.002806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b609376-a1f5-450d-acfb-481d3cc5a4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:32.002911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b609376-a1f5-450d-acfb-481d3cc5a4c5 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:32.004240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 600/1183] 10.222.0.44 () {52 vars in 830 bytes} [Wed Oct 12 12:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:34:32.007673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:32.008862 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:32.107238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133b317-9a35-4af1-bda1-2a3a45a6d544 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:32.199480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:32.200092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:32.200362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:32.200627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] PATH_INFO: `/v3/users/077d9c2cd616436bb37ee8a9b7e4a336` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:32.213155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=077d9c2cd616436bb37ee8a9b7e4a336)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:32.214216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:34:32.214216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 12 12:34:32.214216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: Oct 12 12:34:32.215417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:32.224827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:32.422872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077d9c2cd616436bb37ee8a9b7e4a336'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:32.446145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077d9c2cd616436bb37ee8a9b7e4a336'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:32.556524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077d9c2cd616436bb37ee8a9b7e4a336'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:32.557138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077d9c2cd616436bb37ee8a9b7e4a336'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:32.572297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077d9c2cd616436bb37ee8a9b7e4a336'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:32.578285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 584/1184] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:34:32 2022] DELETE /identity/v3/users/077d9c2cd616436bb37ee8a9b7e4a336 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:34:32.582702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193125f-4ffe-47ba-894d-a9a0e3d2f59a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:32.584927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193125f-4ffe-47ba-894d-a9a0e3d2f59a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:32.584927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193125f-4ffe-47ba-894d-a9a0e3d2f59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:32.640617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5193125f-4ffe-47ba-894d-a9a0e3d2f59a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:32.641861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5193125f-4ffe-47ba-894d-a9a0e3d2f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:32.717006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:32.729102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 601/1185] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:32.739488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:32.745403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:32.842352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430c1bb-e5a1-4324-b99a-edc0739bc0c9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:32.896542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:32.915727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86359420-e1d3-4a6e-baeb-413a45f5efe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:32.916340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86359420-e1d3-4a6e-baeb-413a45f5efe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:32.916620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86359420-e1d3-4a6e-baeb-413a45f5efe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:32.916892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86359420-e1d3-4a6e-baeb-413a45f5efe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:32.919171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-86359420-e1d3-4a6e-baeb-413a45f5efe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:33.026282 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86359420-e1d3-4a6e-baeb-413a45f5efe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:33.028180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86359420-e1d3-4a6e-baeb-413a45f5efe5 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:33.046996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 585/1186] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:33.597142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193125f-4ffe-47ba-894d-a9a0e3d2f59a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:33.597142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5193125f-4ffe-47ba-894d-a9a0e3d2f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:33.608827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193125f-4ffe-47ba-894d-a9a0e3d2f59a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:33.711064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:33.711702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:33.711977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:33.712243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] PATH_INFO: `/v3/projects/2f71b87e82764dd7ad04da030d1e514e` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:33.721957 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f71b87e82764dd7ad04da030d1e514e)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:33.723066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:34:33.723066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:34:33.723066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:34:33.724348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:33.854102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f71b87e82764dd7ad04da030d1e514e', 'request_id': 'req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:33.860072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f71b87e82764dd7ad04da030d1e514e', 'request_id': 'req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:33.874186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f71b87e82764dd7ad04da030d1e514e', 'request_id': 'req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:33.953179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] The token cache is being invalidate because project 2f71b87e82764dd7ad04da030d1e514e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:34:33.953462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:33.953997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a25188a3-e97c-4d9f-b1ec-3e1e4595973b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:33.955326 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 602/1187] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:34:33 2022] DELETE /identity/v3/projects/2f71b87e82764dd7ad04da030d1e514e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:43.044620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=62 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:43.951188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:03.892518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697d7cf-7a0d-4449-98c5-2c3889e7f90a None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:03.893123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697d7cf-7a0d-4449-98c5-2c3889e7f90a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:03.893123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697d7cf-7a0d-4449-98c5-2c3889e7f90a None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:03.894497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 586/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:36:03.899230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0769176-8e4e-4d72-977e-3f9eab177d6a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:03.899678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0769176-8e4e-4d72-977e-3f9eab177d6a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:03.899778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0769176-8e4e-4d72-977e-3f9eab177d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:03.980428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a0769176-8e4e-4d72-977e-3f9eab177d6a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:03.981626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a0769176-8e4e-4d72-977e-3f9eab177d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:04.083947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 603/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:04.236444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697d7cf-7a0d-4449-98c5-2c3889e7f90a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:04.238912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7697d7cf-7a0d-4449-98c5-2c3889e7f90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:04.356346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697d7cf-7a0d-4449-98c5-2c3889e7f90a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:04.362644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecec25-9111-49ec-823e-3f9edd33ce63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:04.363236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecec25-9111-49ec-823e-3f9edd33ce63 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:04.363511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecec25-9111-49ec-823e-3f9edd33ce63 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:04.363779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecec25-9111-49ec-823e-3f9edd33ce63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:04.371625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecec25-9111-49ec-823e-3f9edd33ce63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:04.373206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecec25-9111-49ec-823e-3f9edd33ce63 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:04.382398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 587/1190] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:05.078824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740028a9-33de-495d-8d27-95294ecfa46d None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:05.079275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740028a9-33de-495d-8d27-95294ecfa46d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:05.079422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740028a9-33de-495d-8d27-95294ecfa46d None None] PATH_INFO: `/v3` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:05.080705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 604/1191] 10.222.0.39 () {54 vars in 925 bytes} [Wed Oct 12 12:36:05 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:36:05.087118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59ebcd-6739-4d02-8ad4-004d7be4d7d4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:05.087541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59ebcd-6739-4d02-8ad4-004d7be4d7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:05.087705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59ebcd-6739-4d02-8ad4-004d7be4d7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:05.147326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0d59ebcd-6739-4d02-8ad4-004d7be4d7d4 None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:05.148528 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0d59ebcd-6739-4d02-8ad4-004d7be4d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:05.215613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:05.221133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 588/1192] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Oct 12 12:36:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:36:05.459186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452fe04-262e-4e6c-99fd-a44a024b473a None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:05.459419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452fe04-262e-4e6c-99fd-a44a024b473a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:05.459643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452fe04-262e-4e6c-99fd-a44a024b473a None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:05.460997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 605/1193] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:36:05.466952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b969fe5-3d0d-42c9-9356-8446975985e2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:05.467269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b969fe5-3d0d-42c9-9356-8446975985e2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:05.467550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b969fe5-3d0d-42c9-9356-8446975985e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:05.487749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5b969fe5-3d0d-42c9-9356-8446975985e2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:05.488948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5b969fe5-3d0d-42c9-9356-8446975985e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:05.496711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 589/1194] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:05.646139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452fe04-262e-4e6c-99fd-a44a024b473a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:05.660771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452fe04-262e-4e6c-99fd-a44a024b473a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:05.666044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2bd29-96b0-4464-892d-b59534c64e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:05.666630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2bd29-96b0-4464-892d-b59534c64e4c service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:05.666908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2bd29-96b0-4464-892d-b59534c64e4c service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:05.667172 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2bd29-96b0-4464-892d-b59534c64e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:05.675035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2bd29-96b0-4464-892d-b59534c64e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:05.676589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2bd29-96b0-4464-892d-b59534c64e4c service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:05.685642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 606/1195] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:13.931851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c9372-47c1-42d1-84f3-d0ee289ebd07 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:13.932322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c9372-47c1-42d1-84f3-d0ee289ebd07 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:13.932420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c9372-47c1-42d1-84f3-d0ee289ebd07 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:13.933789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 590/1196] 10.222.0.44 () {54 vars in 961 bytes} [Wed Oct 12 12:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:36:13.938590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439aec1-c413-4594-bed2-eac274fd64dd None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:13.938998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439aec1-c413-4594-bed2-eac274fd64dd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:13.939136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439aec1-c413-4594-bed2-eac274fd64dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:13.960516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-4439aec1-c413-4594-bed2-eac274fd64dd None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:13.961703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4439aec1-c413-4594-bed2-eac274fd64dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:13.970742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 607/1197] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Oct 12 12:36:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:13.977958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c9372-47c1-42d1-84f3-d0ee289ebd07 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:13.991469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c9372-47c1-42d1-84f3-d0ee289ebd07 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:13.996379 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01571a48-6897-4536-9bd6-679acfbb6920 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:13.996992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01571a48-6897-4536-9bd6-679acfbb6920 service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:13.997272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01571a48-6897-4536-9bd6-679acfbb6920 service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:13.997545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01571a48-6897-4536-9bd6-679acfbb6920 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:14.005646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01571a48-6897-4536-9bd6-679acfbb6920 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:14.007358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01571a48-6897-4536-9bd6-679acfbb6920 service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:14.017383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 591/1198] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:17.135661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139f44-474b-4b87-8ba4-738c4bdcac58 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:17.136193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139f44-474b-4b87-8ba4-738c4bdcac58 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:17.136314 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139f44-474b-4b87-8ba4-738c4bdcac58 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:17.137599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 608/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:36:17.144385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690ed8e-2d1e-48d9-8764-692c28c40337 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:17.144853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690ed8e-2d1e-48d9-8764-692c28c40337 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:17.144961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690ed8e-2d1e-48d9-8764-692c28c40337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:17.164960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f690ed8e-2d1e-48d9-8764-692c28c40337 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:17.166197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f690ed8e-2d1e-48d9-8764-692c28c40337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:17.174108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 592/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:17.325663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62139f44-474b-4b87-8ba4-738c4bdcac58 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:17.340615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62139f44-474b-4b87-8ba4-738c4bdcac58 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:17.346381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697aaf5-8d2c-426f-9a31-8dce93507960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:17.346966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697aaf5-8d2c-426f-9a31-8dce93507960 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:17.347245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697aaf5-8d2c-426f-9a31-8dce93507960 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:17.347514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697aaf5-8d2c-426f-9a31-8dce93507960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:17.355603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697aaf5-8d2c-426f-9a31-8dce93507960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:17.357184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697aaf5-8d2c-426f-9a31-8dce93507960 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:17.366941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 609/1201] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:18.427002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683ad92-05e8-4a4a-896e-21d4b9ac2bf2 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:18.427529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683ad92-05e8-4a4a-896e-21d4b9ac2bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:18.427529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683ad92-05e8-4a4a-896e-21d4b9ac2bf2 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:18.428851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 593/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:36:18.434858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dbb8f-b438-4485-a982-2bf41a5396c3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:18.435333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dbb8f-b438-4485-a982-2bf41a5396c3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:18.435439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dbb8f-b438-4485-a982-2bf41a5396c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:18.455223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c05dbb8f-b438-4485-a982-2bf41a5396c3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:18.456415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c05dbb8f-b438-4485-a982-2bf41a5396c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:18.464085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 610/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:18.617151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683ad92-05e8-4a4a-896e-21d4b9ac2bf2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:18.631747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683ad92-05e8-4a4a-896e-21d4b9ac2bf2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:18.637098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c152905-6708-4a37-b157-ec5b5220221c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:18.637674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c152905-6708-4a37-b157-ec5b5220221c service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:18.637972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c152905-6708-4a37-b157-ec5b5220221c service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:18.638246 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c152905-6708-4a37-b157-ec5b5220221c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:18.646627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c152905-6708-4a37-b157-ec5b5220221c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:18.648196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c152905-6708-4a37-b157-ec5b5220221c service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:18.657329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 594/1204] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:22.760053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1bbb2-76d2-4881-98b1-9449c4785e44 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:22.760565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1bbb2-76d2-4881-98b1-9449c4785e44 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:22.760565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1bbb2-76d2-4881-98b1-9449c4785e44 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:22.761883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 611/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:36:22.767655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c91c-a599-4993-852d-f7ff8b49243c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:22.768062 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c91c-a599-4993-852d-f7ff8b49243c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:22.768202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2c91c-a599-4993-852d-f7ff8b49243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:22.787717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1fa2c91c-a599-4993-852d-f7ff8b49243c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:22.788923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1fa2c91c-a599-4993-852d-f7ff8b49243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:22.796712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 595/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:23.095238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1bbb2-76d2-4881-98b1-9449c4785e44 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:23.096978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4fa1bbb2-76d2-4881-98b1-9449c4785e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:23.129786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:23.174062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1bbb2-76d2-4881-98b1-9449c4785e44 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:23.177534 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8484d2-9a3f-4228-86bc-1694e6d2eb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:23.178155 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8484d2-9a3f-4228-86bc-1694e6d2eb9a service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:23.178432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8484d2-9a3f-4228-86bc-1694e6d2eb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:23.178700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8484d2-9a3f-4228-86bc-1694e6d2eb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:23.187308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8484d2-9a3f-4228-86bc-1694e6d2eb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:23.189215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8484d2-9a3f-4228-86bc-1694e6d2eb9a service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:23.198499 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 612/1207] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:25.323082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e606-1502-46c1-8aad-88ec78c92249 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:25.323611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e606-1502-46c1-8aad-88ec78c92249 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:25.323611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e606-1502-46c1-8aad-88ec78c92249 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:25.324962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 596/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:36:25.330457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c492-93c6-4d16-ab6a-5f138636f284 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:25.330868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c492-93c6-4d16-ab6a-5f138636f284 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:25.330990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c492-93c6-4d16-ab6a-5f138636f284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:25.350824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-8e63c492-93c6-4d16-ab6a-5f138636f284 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:25.352006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8e63c492-93c6-4d16-ab6a-5f138636f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:25.360282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 613/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:25.648613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539e606-1502-46c1-8aad-88ec78c92249 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:25.650348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d539e606-1502-46c1-8aad-88ec78c92249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:25.663878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539e606-1502-46c1-8aad-88ec78c92249 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:25.667373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e3df0-fdcf-4426-8c6e-71eccc4c71e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:25.667961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e3df0-fdcf-4426-8c6e-71eccc4c71e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:25.668235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e3df0-fdcf-4426-8c6e-71eccc4c71e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:25.668503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e3df0-fdcf-4426-8c6e-71eccc4c71e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:25.678357 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e3df0-fdcf-4426-8c6e-71eccc4c71e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:25.679750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e3df0-fdcf-4426-8c6e-71eccc4c71e6 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:25.689022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 597/1210] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:29.375091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53341-a226-4a41-b85b-11db6cc35dee None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:29.375611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53341-a226-4a41-b85b-11db6cc35dee None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:29.375611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53341-a226-4a41-b85b-11db6cc35dee None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:29.376941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 614/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:36:29.382275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff8387-2243-483d-96f7-1b0cf0e3bb75 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:29.382830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff8387-2243-483d-96f7-1b0cf0e3bb75 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:29.383137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff8387-2243-483d-96f7-1b0cf0e3bb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:29.402583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-01ff8387-2243-483d-96f7-1b0cf0e3bb75 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:29.403825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-01ff8387-2243-483d-96f7-1b0cf0e3bb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:29.412116 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 598/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:29.720063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53341-a226-4a41-b85b-11db6cc35dee None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:29.722026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e1d53341-a226-4a41-b85b-11db6cc35dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:29.801837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53341-a226-4a41-b85b-11db6cc35dee None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:29.805529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09999e29-f228-44b6-ad9e-a4fbb4386eda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:29.806192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09999e29-f228-44b6-ad9e-a4fbb4386eda service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:29.806478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09999e29-f228-44b6-ad9e-a4fbb4386eda service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:29.806751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09999e29-f228-44b6-ad9e-a4fbb4386eda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:29.816125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09999e29-f228-44b6-ad9e-a4fbb4386eda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:29.817536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09999e29-f228-44b6-ad9e-a4fbb4386eda service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:29.826872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 615/1213] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:31.945410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08be91-e5ce-45bb-a720-785cb15153c1 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:31.946098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08be91-e5ce-45bb-a720-785cb15153c1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:31.946098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08be91-e5ce-45bb-a720-785cb15153c1 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:31.947332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 599/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:36:31.952418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd24542-460b-437e-99e0-a8600a220ae3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:31.952862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd24542-460b-437e-99e0-a8600a220ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:31.952964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd24542-460b-437e-99e0-a8600a220ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:31.973220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-acd24542-460b-437e-99e0-a8600a220ae3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:31.974442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-acd24542-460b-437e-99e0-a8600a220ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:31.982106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 616/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:32.282043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08be91-e5ce-45bb-a720-785cb15153c1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:32.283747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ad08be91-e5ce-45bb-a720-785cb15153c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:32.297239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08be91-e5ce-45bb-a720-785cb15153c1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:32.300715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c93a8-39f5-4fb0-aa58-750a11d5c6a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:32.301327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c93a8-39f5-4fb0-aa58-750a11d5c6a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:32.301606 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c93a8-39f5-4fb0-aa58-750a11d5c6a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:32.301875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c93a8-39f5-4fb0-aa58-750a11d5c6a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:32.310652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c93a8-39f5-4fb0-aa58-750a11d5c6a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:32.312059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c93a8-39f5-4fb0-aa58-750a11d5c6a6 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:32.321726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 600/1216] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:41.967435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=63 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:42.317673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=37 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:55.698881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408fad2-0cf6-4cd8-98df-66470f870e75 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.702845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408fad2-0cf6-4cd8-98df-66470f870e75 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.702845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408fad2-0cf6-4cd8-98df-66470f870e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.704177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ebe41-e671-49f8-9a24-efc7edaccbf4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.704630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ebe41-e671-49f8-9a24-efc7edaccbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.704718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ebe41-e671-49f8-9a24-efc7edaccbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.730526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-8408fad2-0cf6-4cd8-98df-66470f870e75 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:55.732305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8408fad2-0cf6-4cd8-98df-66470f870e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:55.736680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-629ebe41-e671-49f8-9a24-efc7edaccbf4 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:55.738165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-629ebe41-e671-49f8-9a24-efc7edaccbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:55.751964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 617/1217] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:55.752802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 601/1218] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:37:55.754323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7fa5-613a-4899-b527-6146d4d6d6a2 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.754833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7fa5-613a-4899-b527-6146d4d6d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.755333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b7fa5-613a-4899-b527-6146d4d6d6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.762989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb665-9d9d-4e73-9c59-fb5ab077c22a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.763316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb665-9d9d-4e73-9c59-fb5ab077c22a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.763549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabb665-9d9d-4e73-9c59-fb5ab077c22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.790023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-050b7fa5-613a-4899-b527-6146d4d6d6a2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:55.791695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-050b7fa5-613a-4899-b527-6146d4d6d6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:55.794702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5cabb665-9d9d-4e73-9c59-fb5ab077c22a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:55.795895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5cabb665-9d9d-4e73-9c59-fb5ab077c22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:55.805969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 618/1219] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:55.807526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff37df8-a358-431c-8759-c4f6e4a14bf3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.808020 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff37df8-a358-431c-8759-c4f6e4a14bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.808502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff37df8-a358-431c-8759-c4f6e4a14bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.813843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 602/1220] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:37:55.817404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be1f6c-a4da-466b-bc6d-9b89718b5569 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.817693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be1f6c-a4da-466b-bc6d-9b89718b5569 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.818072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be1f6c-a4da-466b-bc6d-9b89718b5569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.842856 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9ff37df8-a358-431c-8759-c4f6e4a14bf3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:55.844665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9ff37df8-a358-431c-8759-c4f6e4a14bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:55.851006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-56be1f6c-a4da-466b-bc6d-9b89718b5569 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:55.852353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-56be1f6c-a4da-466b-bc6d-9b89718b5569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:55.860074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 619/1221] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:55.861291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff37df8-a358-431c-8759-c4f6e4a14bf3 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:55.863869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 603/1222] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:37:55.865646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a6a74-8a9e-4ddd-9e46-3d1a0c76f4fb None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.868774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a6a74-8a9e-4ddd-9e46-3d1a0c76f4fb None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.868774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a6a74-8a9e-4ddd-9e46-3d1a0c76f4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.879535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff37df8-a358-431c-8759-c4f6e4a14bf3 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:55.892282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:55.893870 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.894398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.894888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.895953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:37:55.896477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:55.902531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:55.903838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e50a6a74-8a9e-4ddd-9e46-3d1a0c76f4fb None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:55.905164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e50a6a74-8a9e-4ddd-9e46-3d1a0c76f4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:55.922090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 604/1223] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:55.922566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 620/1224] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:37:55.922958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a6a74-8a9e-4ddd-9e46-3d1a0c76f4fb None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:55.923936 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:55.941462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a6a74-8a9e-4ddd-9e46-3d1a0c76f4fb None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:55.942285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcdf7f-1005-41de-916c-0e3100be2f6e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:55.952415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:55.952967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.953250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.953524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.954355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:37:55.954659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a21b55-0c2b-4618-af25-54846c0b1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:55.954795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:55.955830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a21b55-0c2b-4618-af25-54846c0b1906 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.956115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a21b55-0c2b-4618-af25-54846c0b1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.956386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a21b55-0c2b-4618-af25-54846c0b1906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.956812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:55.957042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a21b55-0c2b-4618-af25-54846c0b1906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:37:55.957351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a21b55-0c2b-4618-af25-54846c0b1906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:55.958414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-73a21b55-0c2b-4618-af25-54846c0b1906 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:37:55.958414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 12 12:37:55.958414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: Oct 12 12:37:55.959394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a21b55-0c2b-4618-af25-54846c0b1906 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:55.978190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 621/1225] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:37:55.979299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:55.982128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 605/1226] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:37:55.983431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7f14b-2c5e-4253-8ac6-5ae7a1d32b09 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:55.983717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7f14b-2c5e-4253-8ac6-5ae7a1d32b09 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:55.983990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7f14b-2c5e-4253-8ac6-5ae7a1d32b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:55.992295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb95e5-0fc7-4194-8e56-5a913140c092 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:55.999291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:55.999905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.000182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.000453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.001089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.002664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:37:56.002664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:37:56.002664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:37:56.004163 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.012916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-22a7f14b-2c5e-4253-8ac6-5ae7a1d32b09 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.014178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22a7f14b-2c5e-4253-8ac6-5ae7a1d32b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.027321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 606/1227] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.028687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48438-fc96-435f-87fd-a1dbdf27a470 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.029216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48438-fc96-435f-87fd-a1dbdf27a470 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.030005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de48438-fc96-435f-87fd-a1dbdf27a470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.050219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8de48438-fc96-435f-87fd-a1dbdf27a470 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.051413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8de48438-fc96-435f-87fd-a1dbdf27a470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.060672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 607/1228] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.060672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de48438-fc96-435f-87fd-a1dbdf27a470 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.070983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de48438-fc96-435f-87fd-a1dbdf27a470 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.077997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec242f-2398-4c40-98ec-39d7786ef5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.078664 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec242f-2398-4c40-98ec-39d7786ef5fc admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.078947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec242f-2398-4c40-98ec-39d7786ef5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.079389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec242f-2398-4c40-98ec-39d7786ef5fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.080653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec242f-2398-4c40-98ec-39d7786ef5fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.082620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec242f-2398-4c40-98ec-39d7786ef5fc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.133249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 622/1229] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:55 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.134171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.186082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 608/1230] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.187674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8fed4-daba-4a8a-9da4-902e0714bf3f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.187966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8fed4-daba-4a8a-9da4-902e0714bf3f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.188244 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8fed4-daba-4a8a-9da4-902e0714bf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.201567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649356f-f87f-42f0-893b-aa5e8cda3d7a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.208574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f2a8fed4-daba-4a8a-9da4-902e0714bf3f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.209724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f2a8fed4-daba-4a8a-9da4-902e0714bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.258476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f45ac7-189f-4b8b-9373-99a38a22c2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.259090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f45ac7-189f-4b8b-9373-99a38a22c2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.259373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f45ac7-189f-4b8b-9373-99a38a22c2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.259651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f45ac7-189f-4b8b-9373-99a38a22c2c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.260281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f45ac7-189f-4b8b-9373-99a38a22c2c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.262348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f45ac7-189f-4b8b-9373-99a38a22c2c6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.269061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 609/1231] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.270415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbca35b-c68f-4d6d-842a-43534a0c8a2a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.270705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbca35b-c68f-4d6d-842a-43534a0c8a2a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.270983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbca35b-c68f-4d6d-842a-43534a0c8a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.290900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ddbca35b-c68f-4d6d-842a-43534a0c8a2a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.292918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ddbca35b-c68f-4d6d-842a-43534a0c8a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.300147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 610/1232] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.300902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbca35b-c68f-4d6d-842a-43534a0c8a2a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.314062 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbca35b-c68f-4d6d-842a-43534a0c8a2a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.319351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.319936 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.320214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.320486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.321125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:37:56.321427 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.323040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.328769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 623/1233] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.330860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069c8b-8008-49f1-9406-5bc89e8330e6 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.331142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069c8b-8008-49f1-9406-5bc89e8330e6 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.331416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069c8b-8008-49f1-9406-5bc89e8330e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.337635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 611/1234] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:37:56.338731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.348856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdee0c8-a98e-4ab2-af35-b1135e674be1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.356417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ea069c8b-8008-49f1-9406-5bc89e8330e6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.357658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ea069c8b-8008-49f1-9406-5bc89e8330e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.367734 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:56.406880 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e3fb1-ae59-461a-a580-6d35b606045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.407534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e3fb1-ae59-461a-a580-6d35b606045b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.407818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e3fb1-ae59-461a-a580-6d35b606045b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.408114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e3fb1-ae59-461a-a580-6d35b606045b admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.408740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e3fb1-ae59-461a-a580-6d35b606045b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.409743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-188e3fb1-ae59-461a-a580-6d35b606045b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:37:56.409743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 12 12:37:56.409743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: Oct 12 12:37:56.410997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e3fb1-ae59-461a-a580-6d35b606045b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.418855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 624/1235] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.420098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7ea14-f07e-42d3-855e-04d10d72bd9b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.420382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7ea14-f07e-42d3-855e-04d10d72bd9b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.420653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7ea14-f07e-42d3-855e-04d10d72bd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.426240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:56.441913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7aa7ea14-f07e-42d3-855e-04d10d72bd9b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.443202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7aa7ea14-f07e-42d3-855e-04d10d72bd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.451052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 625/1236] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.451787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7ea14-f07e-42d3-855e-04d10d72bd9b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.461600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7ea14-f07e-42d3-855e-04d10d72bd9b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.466197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9386a6-feb7-4e79-a654-7e2d73345ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.466779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9386a6-feb7-4e79-a654-7e2d73345ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.467058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9386a6-feb7-4e79-a654-7e2d73345ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.467325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9386a6-feb7-4e79-a654-7e2d73345ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.467944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9386a6-feb7-4e79-a654-7e2d73345ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.469830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9386a6-feb7-4e79-a654-7e2d73345ac2 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.505199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 612/1237] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.506604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db4c9-2809-4165-8505-4f54910fb372 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.506892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db4c9-2809-4165-8505-4f54910fb372 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.507166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db4c9-2809-4165-8505-4f54910fb372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.526511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-134db4c9-2809-4165-8505-4f54910fb372 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.527715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-134db4c9-2809-4165-8505-4f54910fb372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.535600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 613/1238] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.536879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764ee93-cff0-4303-82fb-e4e67b3fe6e1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.537161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764ee93-cff0-4303-82fb-e4e67b3fe6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.537433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764ee93-cff0-4303-82fb-e4e67b3fe6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.557481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5764ee93-cff0-4303-82fb-e4e67b3fe6e1 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.558689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5764ee93-cff0-4303-82fb-e4e67b3fe6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.567111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 614/1239] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.567981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764ee93-cff0-4303-82fb-e4e67b3fe6e1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.577012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764ee93-cff0-4303-82fb-e4e67b3fe6e1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.581596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbf43c-1756-4e42-b960-5d40d0bdc5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.582191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbf43c-1756-4e42-b960-5d40d0bdc5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.582469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbf43c-1756-4e42-b960-5d40d0bdc5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.582741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbf43c-1756-4e42-b960-5d40d0bdc5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.583472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbf43c-1756-4e42-b960-5d40d0bdc5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.585363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbf43c-1756-4e42-b960-5d40d0bdc5d5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.595654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 626/1240] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.597125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9386a6-feb7-4e79-a654-7e2d73345ac2 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.606446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9386a6-feb7-4e79-a654-7e2d73345ac2 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.611839 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ea805-38e0-4e07-b562-c8f5c890e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.612419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea805-38e0-4e07-b562-c8f5c890e1a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.612700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea805-38e0-4e07-b562-c8f5c890e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.612990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ea805-38e0-4e07-b562-c8f5c890e1a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.613599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ea805-38e0-4e07-b562-c8f5c890e1a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.615980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ea805-38e0-4e07-b562-c8f5c890e1a4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.670599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 615/1241] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.670599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbf43c-1756-4e42-b960-5d40d0bdc5d5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.678760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbf43c-1756-4e42-b960-5d40d0bdc5d5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.683634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.684417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.684873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.685262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.686277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:37:56.686580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.688157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.696110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 627/1242] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.696110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be8b1f-ecaa-40e2-9bfb-58a4cc40728b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.696110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be8b1f-ecaa-40e2-9bfb-58a4cc40728b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.696110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be8b1f-ecaa-40e2-9bfb-58a4cc40728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.709259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 616/1243] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:37:56.709259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.714924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-52be8b1f-ecaa-40e2-9bfb-58a4cc40728b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.715573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339b973-a831-476b-908a-cd7a11bfcb85 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.716075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-52be8b1f-ecaa-40e2-9bfb-58a4cc40728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.773240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a371374-fc7b-40dc-adde-ccc9262bb085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.773852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a371374-fc7b-40dc-adde-ccc9262bb085 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.774158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a371374-fc7b-40dc-adde-ccc9262bb085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.774432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a371374-fc7b-40dc-adde-ccc9262bb085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.775063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a371374-fc7b-40dc-adde-ccc9262bb085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.776681 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a371374-fc7b-40dc-adde-ccc9262bb085 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.780771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 628/1244] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:56.780771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be8b1f-ecaa-40e2-9bfb-58a4cc40728b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.789990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 617/1245] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:56.789990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a371374-fc7b-40dc-adde-ccc9262bb085 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.790260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be8b1f-ecaa-40e2-9bfb-58a4cc40728b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.794197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.794812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.795095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.795370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.796011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:37:56.796329 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.797496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a371374-fc7b-40dc-adde-ccc9262bb085 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.797885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.802639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.803209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.803484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.803760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.804385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:37:56.804681 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.806266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.810127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 629/1246] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:37:56.811004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.819312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 618/1247] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:37:56.820015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.821158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468f58d-9a7b-4d5b-aaa0-e8f21577ca85 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.826123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64176f0-bdc6-49a3-8769-061b54b25cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.826712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64176f0-bdc6-49a3-8769-061b54b25cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.826992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64176f0-bdc6-49a3-8769-061b54b25cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.827264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64176f0-bdc6-49a3-8769-061b54b25cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.827896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64176f0-bdc6-49a3-8769-061b54b25cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.829475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64176f0-bdc6-49a3-8769-061b54b25cc4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.831006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d949be-e053-451e-a47a-c1869949e9cf admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.835574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b600564-b77c-4dd7-bd53-8064962869c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.836157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b600564-b77c-4dd7-bd53-8064962869c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.836433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b600564-b77c-4dd7-bd53-8064962869c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.836706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b600564-b77c-4dd7-bd53-8064962869c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.837364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b600564-b77c-4dd7-bd53-8064962869c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.838947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b600564-b77c-4dd7-bd53-8064962869c4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.840106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 630/1248] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:56.840943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64176f0-bdc6-49a3-8769-061b54b25cc4 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.851158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64176f0-bdc6-49a3-8769-061b54b25cc4 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.855237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 619/1249] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:56.855237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b600564-b77c-4dd7-bd53-8064962869c4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.858370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a33c1-f677-4d90-b834-1cd35e310e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.858953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a33c1-f677-4d90-b834-1cd35e310e83 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.859232 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a33c1-f677-4d90-b834-1cd35e310e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.859501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a33c1-f677-4d90-b834-1cd35e310e83 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.860121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a33c1-f677-4d90-b834-1cd35e310e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.862657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a33c1-f677-4d90-b834-1cd35e310e83 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.863151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b600564-b77c-4dd7-bd53-8064962869c4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.867926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3aa5-24bd-4752-8045-66972b0bcba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.868628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3aa5-24bd-4752-8045-66972b0bcba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.868912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3aa5-24bd-4752-8045-66972b0bcba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.869184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b3aa5-24bd-4752-8045-66972b0bcba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.869802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b3aa5-24bd-4752-8045-66972b0bcba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.871690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b3aa5-24bd-4752-8045-66972b0bcba3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.947267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 631/1250] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.947267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a33c1-f677-4d90-b834-1cd35e310e83 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.955253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a33c1-f677-4d90-b834-1cd35e310e83 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:56.959925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:56.960496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.960778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.961053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.961687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:37:56.962021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:56.963600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:56.973146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 620/1251] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:56.973146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3aa5-24bd-4752-8045-66972b0bcba3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.980929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 632/1252] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:37:56.981733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:56.994533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40594781-3538-4ab6-b6a0-198362a1810b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.005889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:57.032595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b3aa5-24bd-4752-8045-66972b0bcba3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.039570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f1b50-72ec-4bd2-9505-966c6254374d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.040225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f1b50-72ec-4bd2-9505-966c6254374d admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.040574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f1b50-72ec-4bd2-9505-966c6254374d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.040850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f1b50-72ec-4bd2-9505-966c6254374d admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.041799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f1b50-72ec-4bd2-9505-966c6254374d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.043479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f1b50-72ec-4bd2-9505-966c6254374d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.045575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:57.051991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdf200-4c94-49e9-9d69-18f6e27dc4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.052591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf200-4c94-49e9-9d69-18f6e27dc4ed admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.052867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf200-4c94-49e9-9d69-18f6e27dc4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.053135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdf200-4c94-49e9-9d69-18f6e27dc4ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.053754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdf200-4c94-49e9-9d69-18f6e27dc4ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.054837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 621/1253] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:57.055665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f1b50-72ec-4bd2-9505-966c6254374d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.055791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdf200-4c94-49e9-9d69-18f6e27dc4ed admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.070090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f1b50-72ec-4bd2-9505-966c6254374d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.076150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2f2f8-5011-405e-ab17-ce46cdccd70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.076756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2f2f8-5011-405e-ab17-ce46cdccd70e admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.077038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2f2f8-5011-405e-ab17-ce46cdccd70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.077311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2f2f8-5011-405e-ab17-ce46cdccd70e admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.077966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2f2f8-5011-405e-ab17-ce46cdccd70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.079864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2f2f8-5011-405e-ab17-ce46cdccd70e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.157135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 633/1254] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/projects => generated 393 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:57.158042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdf200-4c94-49e9-9d69-18f6e27dc4ed admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.178342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 622/1255] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:57 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:57.179371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2f2f8-5011-405e-ab17-ce46cdccd70e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.222717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdf200-4c94-49e9-9d69-18f6e27dc4ed admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.240213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2f2f8-5011-405e-ab17-ce46cdccd70e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.244791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17e889-38bc-43db-911a-306f982f9347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.245388 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e889-38bc-43db-911a-306f982f9347 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.245666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e889-38bc-43db-911a-306f982f9347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.245957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17e889-38bc-43db-911a-306f982f9347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.246600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17e889-38bc-43db-911a-306f982f9347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.248201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17e889-38bc-43db-911a-306f982f9347 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.257924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 623/1256] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:57.258724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17e889-38bc-43db-911a-306f982f9347 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.268515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17e889-38bc-43db-911a-306f982f9347 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.273442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e08f6-704b-44ef-a06c-1ca6d394d654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.274297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e08f6-704b-44ef-a06c-1ca6d394d654 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.274749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e08f6-704b-44ef-a06c-1ca6d394d654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.275030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e08f6-704b-44ef-a06c-1ca6d394d654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.275785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e08f6-704b-44ef-a06c-1ca6d394d654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.278601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e08f6-704b-44ef-a06c-1ca6d394d654 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.284044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df238018-5709-4712-b875-fec01c5b7008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.284753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df238018-5709-4712-b875-fec01c5b7008 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.285026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df238018-5709-4712-b875-fec01c5b7008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.285293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df238018-5709-4712-b875-fec01c5b7008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.285928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df238018-5709-4712-b875-fec01c5b7008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.287542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df238018-5709-4712-b875-fec01c5b7008 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.288873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 624/1257] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:57.289723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e08f6-704b-44ef-a06c-1ca6d394d654 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.299363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e08f6-704b-44ef-a06c-1ca6d394d654 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.303832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 634/1258] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:57.303832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df238018-5709-4712-b875-fec01c5b7008 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.304383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8471-1b7f-44fc-b5e9-98ae35937043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.305000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8471-1b7f-44fc-b5e9-98ae35937043 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.305284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8471-1b7f-44fc-b5e9-98ae35937043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.305565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8471-1b7f-44fc-b5e9-98ae35937043 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.306225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e8471-1b7f-44fc-b5e9-98ae35937043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.308345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e8471-1b7f-44fc-b5e9-98ae35937043 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.311219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df238018-5709-4712-b875-fec01c5b7008 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.315690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcd759-ec81-4009-a98c-b4c0946d7a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.316265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcd759-ec81-4009-a98c-b4c0946d7a90 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.316540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcd759-ec81-4009-a98c-b4c0946d7a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.316808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcd759-ec81-4009-a98c-b4c0946d7a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.317423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcd759-ec81-4009-a98c-b4c0946d7a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.319302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcd759-ec81-4009-a98c-b4c0946d7a90 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.401821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 625/1259] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:57 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:57.402719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8471-1b7f-44fc-b5e9-98ae35937043 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.412611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8471-1b7f-44fc-b5e9-98ae35937043 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.416065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 635/1260] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:57 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:57.416065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcd759-ec81-4009-a98c-b4c0946d7a90 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.438620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:57.475088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ecab6-e475-4cc4-a282-ce44a091e4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.475671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcd759-ec81-4009-a98c-b4c0946d7a90 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.475819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ecab6-e475-4cc4-a282-ce44a091e4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.476016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ecab6-e475-4cc4-a282-ce44a091e4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.476339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ecab6-e475-4cc4-a282-ce44a091e4e5 admin admin] PATH_INFO: `/v3/projects/c056ad05ef9843688f08b7e9f25e1138/users/df8da1fe608d4dd6acb6b7298ca56d45/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.480439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcfe0a-4066-4e4c-9365-42642e18c4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.481042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcfe0a-4066-4e4c-9365-42642e18c4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.481320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcfe0a-4066-4e4c-9365-42642e18c4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.481595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcfe0a-4066-4e4c-9365-42642e18c4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.482275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcfe0a-4066-4e4c-9365-42642e18c4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.484200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcfe0a-4066-4e4c-9365-42642e18c4f6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.491393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ecab6-e475-4cc4-a282-ce44a091e4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c056ad05ef9843688f08b7e9f25e1138, user_id=df8da1fe608d4dd6acb6b7298ca56d45, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.493996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ecab6-e475-4cc4-a282-ce44a091e4e5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.575968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 636/1261] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:57 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:57.576985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcfe0a-4066-4e4c-9365-42642e18c4f6 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.587091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcfe0a-4066-4e4c-9365-42642e18c4f6 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.591695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956b79f-0e87-47a1-8ecf-2eeb9a9ba4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.592252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956b79f-0e87-47a1-8ecf-2eeb9a9ba4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.592534 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956b79f-0e87-47a1-8ecf-2eeb9a9ba4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.592805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956b79f-0e87-47a1-8ecf-2eeb9a9ba4c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.593410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956b79f-0e87-47a1-8ecf-2eeb9a9ba4c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.595335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956b79f-0e87-47a1-8ecf-2eeb9a9ba4c0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.604664 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 626/1262] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:57 2022] PUT /identity/v3/projects/c056ad05ef9843688f08b7e9f25e1138/users/df8da1fe608d4dd6acb6b7298ca56d45/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:57.605597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ecab6-e475-4cc4-a282-ce44a091e4e5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.634488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:57.665452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ecab6-e475-4cc4-a282-ce44a091e4e5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.670058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafde227-35c8-4fd6-b521-18e934eb544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.670708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafde227-35c8-4fd6-b521-18e934eb544e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.670987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafde227-35c8-4fd6-b521-18e934eb544e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.671260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafde227-35c8-4fd6-b521-18e934eb544e admin admin] PATH_INFO: `/v3/projects/fb8207ad9abf449b99382a2586d4588a/users/39543aedd1834f68a9d0a4ef174822c2/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.679723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 637/1263] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:57 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:57.680833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956b79f-0e87-47a1-8ecf-2eeb9a9ba4c0 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.685764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafde227-35c8-4fd6-b521-18e934eb544e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8207ad9abf449b99382a2586d4588a, user_id=39543aedd1834f68a9d0a4ef174822c2, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.688465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafde227-35c8-4fd6-b521-18e934eb544e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.691103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956b79f-0e87-47a1-8ecf-2eeb9a9ba4c0 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.695756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19457ddf-f56b-4e91-b9e6-9f130ce20802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.696335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19457ddf-f56b-4e91-b9e6-9f130ce20802 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.696612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19457ddf-f56b-4e91-b9e6-9f130ce20802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.696882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19457ddf-f56b-4e91-b9e6-9f130ce20802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.697507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19457ddf-f56b-4e91-b9e6-9f130ce20802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.699098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19457ddf-f56b-4e91-b9e6-9f130ce20802 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.710238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 638/1264] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:57.711107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19457ddf-f56b-4e91-b9e6-9f130ce20802 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.720659 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19457ddf-f56b-4e91-b9e6-9f130ce20802 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.725301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d182-3984-4dc4-a5bd-fdb9eeaf54f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.725913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d182-3984-4dc4-a5bd-fdb9eeaf54f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.726212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d182-3984-4dc4-a5bd-fdb9eeaf54f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.726478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d182-3984-4dc4-a5bd-fdb9eeaf54f3 admin admin] PATH_INFO: `/v3/projects/677a44a0d7bb436f83c23a8e74cffa60/users/df838ad816744832baa283dabe540394/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.741277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045d182-3984-4dc4-a5bd-fdb9eeaf54f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=677a44a0d7bb436f83c23a8e74cffa60, user_id=df838ad816744832baa283dabe540394, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.745843 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045d182-3984-4dc4-a5bd-fdb9eeaf54f3 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.772024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 627/1265] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:57 2022] PUT /identity/v3/projects/fb8207ad9abf449b99382a2586d4588a/users/39543aedd1834f68a9d0a4ef174822c2/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:57.772958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafde227-35c8-4fd6-b521-18e934eb544e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.833003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafde227-35c8-4fd6-b521-18e934eb544e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.837720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c0877-ac2e-4508-b44f-870f50b0790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.838372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c0877-ac2e-4508-b44f-870f50b0790f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.838648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c0877-ac2e-4508-b44f-870f50b0790f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.838917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c0877-ac2e-4508-b44f-870f50b0790f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.839541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c0877-ac2e-4508-b44f-870f50b0790f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.841108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c0877-ac2e-4508-b44f-870f50b0790f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.850897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 628/1266] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:57.851799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c0877-ac2e-4508-b44f-870f50b0790f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.858771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 639/1267] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:57 2022] PUT /identity/v3/projects/677a44a0d7bb436f83c23a8e74cffa60/users/df838ad816744832baa283dabe540394/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:57.859590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d182-3984-4dc4-a5bd-fdb9eeaf54f3 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.861599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c0877-ac2e-4508-b44f-870f50b0790f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.918900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981667-3200-4859-97c0-b88c39ce316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.919505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981667-3200-4859-97c0-b88c39ce316d admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.919790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981667-3200-4859-97c0-b88c39ce316d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.920075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981667-3200-4859-97c0-b88c39ce316d admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.920683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46981667-3200-4859-97c0-b88c39ce316d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.922085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d182-3984-4dc4-a5bd-fdb9eeaf54f3 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.922631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46981667-3200-4859-97c0-b88c39ce316d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.927605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cecfee-00b1-47ab-93f7-33d840e82e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.928315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cecfee-00b1-47ab-93f7-33d840e82e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.928596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cecfee-00b1-47ab-93f7-33d840e82e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.928865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cecfee-00b1-47ab-93f7-33d840e82e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.929501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cecfee-00b1-47ab-93f7-33d840e82e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.931105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cecfee-00b1-47ab-93f7-33d840e82e79 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.942134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 640/1268] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:57.946684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e2afc-2408-4413-8b31-cb2f59a40653 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.946684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e2afc-2408-4413-8b31-cb2f59a40653 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.946684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e2afc-2408-4413-8b31-cb2f59a40653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.961563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a38e2afc-2408-4413-8b31-cb2f59a40653 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:57.962781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a38e2afc-2408-4413-8b31-cb2f59a40653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:57.970142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 641/1269] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:57.970998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e2afc-2408-4413-8b31-cb2f59a40653 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:57.980040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e2afc-2408-4413-8b31-cb2f59a40653 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:57.984490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e64db-1150-4cb2-a0aa-7fc2b9d97478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:57.986864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e64db-1150-4cb2-a0aa-7fc2b9d97478 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:57.986864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e64db-1150-4cb2-a0aa-7fc2b9d97478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:57.986864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e64db-1150-4cb2-a0aa-7fc2b9d97478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:57.986864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e64db-1150-4cb2-a0aa-7fc2b9d97478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:57.988298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e64db-1150-4cb2-a0aa-7fc2b9d97478 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:57.998218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 642/1270] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:57.998891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e64db-1150-4cb2-a0aa-7fc2b9d97478 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.008702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e64db-1150-4cb2-a0aa-7fc2b9d97478 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.013277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f44364-7475-47bf-a893-0cb5d74e55b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.013887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f44364-7475-47bf-a893-0cb5d74e55b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.014237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f44364-7475-47bf-a893-0cb5d74e55b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.014533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f44364-7475-47bf-a893-0cb5d74e55b1 admin admin] PATH_INFO: `/v3/projects/08492244b10d4219a38e24a648a90c56/users/4eb284a8d97d42dfb361595571ad531c/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.028898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f44364-7475-47bf-a893-0cb5d74e55b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08492244b10d4219a38e24a648a90c56, user_id=4eb284a8d97d42dfb361595571ad531c, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.032154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f44364-7475-47bf-a893-0cb5d74e55b1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.037322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 629/1271] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:57 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:58.037322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb8546-e43b-4a1a-8209-477c6f2c760b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.037322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb8546-e43b-4a1a-8209-477c6f2c760b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.037322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb8546-e43b-4a1a-8209-477c6f2c760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.054760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-45cb8546-e43b-4a1a-8209-477c6f2c760b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:58.055941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-45cb8546-e43b-4a1a-8209-477c6f2c760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:58.065210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 630/1272] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:58.065210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb8546-e43b-4a1a-8209-477c6f2c760b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.074524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb8546-e43b-4a1a-8209-477c6f2c760b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.079408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a90a73-b58d-4cc1-8534-c3718983daa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.080071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90a73-b58d-4cc1-8534-c3718983daa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.080350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90a73-b58d-4cc1-8534-c3718983daa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.080619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90a73-b58d-4cc1-8534-c3718983daa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.081256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a90a73-b58d-4cc1-8534-c3718983daa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.082876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a90a73-b58d-4cc1-8534-c3718983daa5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.093032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 631/1273] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:58.094272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cd3f-80e6-48a3-a831-68d710ae7f09 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.094555 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cd3f-80e6-48a3-a831-68d710ae7f09 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.094823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cd3f-80e6-48a3-a831-68d710ae7f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.113238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9fe4cd3f-80e6-48a3-a831-68d710ae7f09 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:58.114459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9fe4cd3f-80e6-48a3-a831-68d710ae7f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:58.116361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 643/1274] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:57 2022] PUT /identity/v3/projects/08492244b10d4219a38e24a648a90c56/users/4eb284a8d97d42dfb361595571ad531c/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:58.117254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f44364-7475-47bf-a893-0cb5d74e55b1 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.136836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:58.137033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:58.183080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 632/1275] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:58.183992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4cd3f-80e6-48a3-a831-68d710ae7f09 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.186701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f44364-7475-47bf-a893-0cb5d74e55b1 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.192760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0863a-defa-40ac-b275-0ed504ea4e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.193478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0863a-defa-40ac-b275-0ed504ea4e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.202322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0863a-defa-40ac-b275-0ed504ea4e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.202322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0863a-defa-40ac-b275-0ed504ea4e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.202322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0863a-defa-40ac-b275-0ed504ea4e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.202322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0863a-defa-40ac-b275-0ed504ea4e4a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.206516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 644/1276] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:58.207298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0863a-defa-40ac-b275-0ed504ea4e4a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.216611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0863a-defa-40ac-b275-0ed504ea4e4a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.221438 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaf437-224b-4cd7-9f1e-b8271cc128ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.222335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf437-224b-4cd7-9f1e-b8271cc128ce admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.222674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf437-224b-4cd7-9f1e-b8271cc128ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.223079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaf437-224b-4cd7-9f1e-b8271cc128ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.224103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf437-224b-4cd7-9f1e-b8271cc128ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.225694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaf437-224b-4cd7-9f1e-b8271cc128ce admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.235692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 645/1277] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:58.237256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaf437-224b-4cd7-9f1e-b8271cc128ce admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.242179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4cd3f-80e6-48a3-a831-68d710ae7f09 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.246067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2ffeb-cc3c-45bb-ba0e-c6a3882c4b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.246672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ffeb-cc3c-45bb-ba0e-c6a3882c4b49 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.246950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ffeb-cc3c-45bb-ba0e-c6a3882c4b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.247200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaf437-224b-4cd7-9f1e-b8271cc128ce admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.247302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2ffeb-cc3c-45bb-ba0e-c6a3882c4b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.251859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65439a68-7d35-433f-bdab-e2d77b5d9d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.252959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65439a68-7d35-433f-bdab-e2d77b5d9d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.253236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65439a68-7d35-433f-bdab-e2d77b5d9d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.253508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65439a68-7d35-433f-bdab-e2d77b5d9d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.254167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65439a68-7d35-433f-bdab-e2d77b5d9d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.255719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65439a68-7d35-433f-bdab-e2d77b5d9d73 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.257699 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2ffeb-cc3c-45bb-ba0e-c6a3882c4b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.259147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2ffeb-cc3c-45bb-ba0e-c6a3882c4b49 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.266429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 646/1278] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:58.267687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65439a68-7d35-433f-bdab-e2d77b5d9d73 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.268478 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 633/1279] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:37:58.269235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2ffeb-cc3c-45bb-ba0e-c6a3882c4b49 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.277550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65439a68-7d35-433f-bdab-e2d77b5d9d73 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.278221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2ffeb-cc3c-45bb-ba0e-c6a3882c4b49 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.281294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de567269-1bc2-460a-b99a-8c651fa5f3fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.281877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de567269-1bc2-460a-b99a-8c651fa5f3fc service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.282198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de567269-1bc2-460a-b99a-8c651fa5f3fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.282466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de567269-1bc2-460a-b99a-8c651fa5f3fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.284030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e7566-b2d3-4d27-bc9a-ae35cb1a9bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.284675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e7566-b2d3-4d27-bc9a-ae35cb1a9bec admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.284951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e7566-b2d3-4d27-bc9a-ae35cb1a9bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.285223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e7566-b2d3-4d27-bc9a-ae35cb1a9bec admin admin] PATH_INFO: `/v3/projects/d329d951b42740b88a3782cd08961ffb/users/8056f7de10564ccf9bdfcd035b721532/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.293239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de567269-1bc2-460a-b99a-8c651fa5f3fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.294723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de567269-1bc2-460a-b99a-8c651fa5f3fc service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.300703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e7566-b2d3-4d27-bc9a-ae35cb1a9bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d329d951b42740b88a3782cd08961ffb, user_id=8056f7de10564ccf9bdfcd035b721532, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.303526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e7566-b2d3-4d27-bc9a-ae35cb1a9bec admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.304967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 647/1280] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:37:58.306324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43fe50-c771-487a-a3fc-abd910089a7a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.306615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43fe50-c771-487a-a3fc-abd910089a7a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.307015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43fe50-c771-487a-a3fc-abd910089a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.333541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-be43fe50-c771-487a-a3fc-abd910089a7a None None] MFA Rules not processed for user `4eb284a8d97d42dfb361595571ad531c`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:58.334788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-be43fe50-c771-487a-a3fc-abd910089a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:58.383086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 634/1281] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:58 2022] PUT /identity/v3/projects/d329d951b42740b88a3782cd08961ffb/users/8056f7de10564ccf9bdfcd035b721532/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:58.392870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e7566-b2d3-4d27-bc9a-ae35cb1a9bec admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.417402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 648/1282] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:58.418109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43fe50-c771-487a-a3fc-abd910089a7a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.431529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43fe50-c771-487a-a3fc-abd910089a7a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.461360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e7566-b2d3-4d27-bc9a-ae35cb1a9bec admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.467276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ccad5-1e89-4403-b47d-bc7d1dd5a654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.467988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ccad5-1e89-4403-b47d-bc7d1dd5a654 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.468407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ccad5-1e89-4403-b47d-bc7d1dd5a654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.469887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ccad5-1e89-4403-b47d-bc7d1dd5a654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.495872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3213e3-7494-4a02-a86c-af5f06751012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.496457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3213e3-7494-4a02-a86c-af5f06751012 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.496828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3213e3-7494-4a02-a86c-af5f06751012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.497168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3213e3-7494-4a02-a86c-af5f06751012 admin admin] PATH_INFO: `/v3/projects/d62398af777d4d269977b1a5f1f37080/users/19d863a45ac14c50b940fbc217b70929/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.507825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:58.515862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3213e3-7494-4a02-a86c-af5f06751012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62398af777d4d269977b1a5f1f37080, user_id=19d863a45ac14c50b940fbc217b70929, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.519504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3213e3-7494-4a02-a86c-af5f06751012 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.525356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:58.538358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ccad5-1e89-4403-b47d-bc7d1dd5a654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.539782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ccad5-1e89-4403-b47d-bc7d1dd5a654 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.550493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 635/1283] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:37:58.551273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ccad5-1e89-4403-b47d-bc7d1dd5a654 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.561022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ccad5-1e89-4403-b47d-bc7d1dd5a654 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.566123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399037a0-8be0-486e-bc73-9e90e18f822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.575372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399037a0-8be0-486e-bc73-9e90e18f822d admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.575372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399037a0-8be0-486e-bc73-9e90e18f822d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.575372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399037a0-8be0-486e-bc73-9e90e18f822d admin admin] PATH_INFO: `/v3/projects/2a372db0342f425cbd7dedccf68e7fd4/users/b65fe96949fe4f87970dc5657b2125f5/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.582728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399037a0-8be0-486e-bc73-9e90e18f822d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a372db0342f425cbd7dedccf68e7fd4, user_id=b65fe96949fe4f87970dc5657b2125f5, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.585441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399037a0-8be0-486e-bc73-9e90e18f822d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.605921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 649/1284] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:58 2022] PUT /identity/v3/projects/d62398af777d4d269977b1a5f1f37080/users/19d863a45ac14c50b940fbc217b70929/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:58.605921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3213e3-7494-4a02-a86c-af5f06751012 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.670162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 636/1285] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:58 2022] PUT /identity/v3/projects/2a372db0342f425cbd7dedccf68e7fd4/users/b65fe96949fe4f87970dc5657b2125f5/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:58.670162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b3d25-5a89-42c4-8384-15c693455804 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.670162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b3d25-5a89-42c4-8384-15c693455804 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.670162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b3d25-5a89-42c4-8384-15c693455804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.682338 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3213e3-7494-4a02-a86c-af5f06751012 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.702904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-043b3d25-5a89-42c4-8384-15c693455804 None None] MFA Rules not processed for user `8056f7de10564ccf9bdfcd035b721532`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:58.709174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-043b3d25-5a89-42c4-8384-15c693455804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:58.752278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881768de-e6de-4c6a-bfc3-39d8e800853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.753050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881768de-e6de-4c6a-bfc3-39d8e800853a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.753427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881768de-e6de-4c6a-bfc3-39d8e800853a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.753892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881768de-e6de-4c6a-bfc3-39d8e800853a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.754896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881768de-e6de-4c6a-bfc3-39d8e800853a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.756700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881768de-e6de-4c6a-bfc3-39d8e800853a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.772872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 650/1286] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:58.772872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881768de-e6de-4c6a-bfc3-39d8e800853a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.779753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881768de-e6de-4c6a-bfc3-39d8e800853a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.785471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd08ef8-6e7d-4e9e-9ad5-4e4a64eb0d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.786229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd08ef8-6e7d-4e9e-9ad5-4e4a64eb0d65 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.786795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd08ef8-6e7d-4e9e-9ad5-4e4a64eb0d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.787055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd08ef8-6e7d-4e9e-9ad5-4e4a64eb0d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.787817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd08ef8-6e7d-4e9e-9ad5-4e4a64eb0d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:58.789777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd08ef8-6e7d-4e9e-9ad5-4e4a64eb0d65 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:58.791906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 637/1287] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:58.801269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee9d3-3655-4f9a-a773-5682f12db098 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.801269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee9d3-3655-4f9a-a773-5682f12db098 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.801269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcee9d3-3655-4f9a-a773-5682f12db098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.815980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-cbcee9d3-3655-4f9a-a773-5682f12db098 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:58.817247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-cbcee9d3-3655-4f9a-a773-5682f12db098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:58.828479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 638/1288] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:58.836198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7c30a-2c55-4c56-ba5a-58f4dc2dfa5e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.836198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7c30a-2c55-4c56-ba5a-58f4dc2dfa5e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.836198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7c30a-2c55-4c56-ba5a-58f4dc2dfa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.863418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-29e7c30a-2c55-4c56-ba5a-58f4dc2dfa5e None None] MFA Rules not processed for user `b65fe96949fe4f87970dc5657b2125f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:58.864951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-29e7c30a-2c55-4c56-ba5a-58f4dc2dfa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:58.893552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 651/1289] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:58 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:58.893552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd08ef8-6e7d-4e9e-9ad5-4e4a64eb0d65 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.945018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 639/1290] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:58.946896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7c30a-2c55-4c56-ba5a-58f4dc2dfa5e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:58.974686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd08ef8-6e7d-4e9e-9ad5-4e4a64eb0d65 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:58.975732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7ed1c-fcce-4f9f-a228-a3a219c23ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:58.976754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7ed1c-fcce-4f9f-a228-a3a219c23ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:58.977043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7ed1c-fcce-4f9f-a228-a3a219c23ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:58.977605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7ed1c-fcce-4f9f-a228-a3a219c23ef5 admin admin] PATH_INFO: `/v3/projects/f427efac014644dea26de8e1f49121a4/users/d81df313cba34fdd9717fee325ec8b1e/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:58.993528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:59.000285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:59.000897 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7ed1c-fcce-4f9f-a228-a3a219c23ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f427efac014644dea26de8e1f49121a4, user_id=d81df313cba34fdd9717fee325ec8b1e, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:59.005452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7ed1c-fcce-4f9f-a228-a3a219c23ef5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:59.017476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7c30a-2c55-4c56-ba5a-58f4dc2dfa5e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:59.080911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18d3f7-f0e6-4078-a129-9e60d5ee48fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:59.081538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18d3f7-f0e6-4078-a129-9e60d5ee48fa service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.081819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18d3f7-f0e6-4078-a129-9e60d5ee48fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.082122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18d3f7-f0e6-4078-a129-9e60d5ee48fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.127511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 652/1291] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:58 2022] PUT /identity/v3/projects/f427efac014644dea26de8e1f49121a4/users/d81df313cba34fdd9717fee325ec8b1e/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:59.128470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7ed1c-fcce-4f9f-a228-a3a219c23ef5 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:59.148651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18d3f7-f0e6-4078-a129-9e60d5ee48fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:59.150589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18d3f7-f0e6-4078-a129-9e60d5ee48fa service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:59.163350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 640/1292] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:37:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:37:59.164756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18d3f7-f0e6-4078-a129-9e60d5ee48fa service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:59.183823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18d3f7-f0e6-4078-a129-9e60d5ee48fa service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:59.266382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7ed1c-fcce-4f9f-a228-a3a219c23ef5 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:59.276665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5feb2c7-eb28-4e85-905d-21ad977fa9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:59.277305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feb2c7-eb28-4e85-905d-21ad977fa9fd service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.277624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feb2c7-eb28-4e85-905d-21ad977fa9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.277896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feb2c7-eb28-4e85-905d-21ad977fa9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.292287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100e745-2fd8-4d2e-9a78-b47f0a483f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:59.294045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100e745-2fd8-4d2e-9a78-b47f0a483f0e admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.294045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100e745-2fd8-4d2e-9a78-b47f0a483f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.294045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100e745-2fd8-4d2e-9a78-b47f0a483f0e admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.294327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100e745-2fd8-4d2e-9a78-b47f0a483f0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:59.296351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100e745-2fd8-4d2e-9a78-b47f0a483f0e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:59.350753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5feb2c7-eb28-4e85-905d-21ad977fa9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:59.352617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5feb2c7-eb28-4e85-905d-21ad977fa9fd service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:59.371345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 653/1293] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:37:59.373174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ab449-f5b5-4bcf-a62e-30364a3ec990 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.373584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ab449-f5b5-4bcf-a62e-30364a3ec990 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.373951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ab449-f5b5-4bcf-a62e-30364a3ec990 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.378311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 654/1294] 10.222.0.44 () {54 vars in 961 bytes} [Wed Oct 12 12:37:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:37:59.379768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978528-b5bd-4d0a-843a-febb6274b587 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.380340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978528-b5bd-4d0a-843a-febb6274b587 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.380931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8978528-b5bd-4d0a-843a-febb6274b587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.424323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 641/1295] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:59 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:59.424323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2108f57-ee51-4e94-b42f-540334b3808b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.424323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2108f57-ee51-4e94-b42f-540334b3808b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.424323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2108f57-ee51-4e94-b42f-540334b3808b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.436976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-f8978528-b5bd-4d0a-843a-febb6274b587 None None] MFA Rules not processed for user `d81df313cba34fdd9717fee325ec8b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:59.438408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f8978528-b5bd-4d0a-843a-febb6274b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:59.521407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b2108f57-ee51-4e94-b42f-540334b3808b None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:59.524434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b2108f57-ee51-4e94-b42f-540334b3808b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:59.555702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 655/1296] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:37:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:59.555702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8978528-b5bd-4d0a-843a-febb6274b587 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:59.575989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8978528-b5bd-4d0a-843a-febb6274b587 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:59.583801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da409de2-171e-4aab-8b6b-6c4a4aaf0b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:59.585094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409de2-171e-4aab-8b6b-6c4a4aaf0b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.585213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409de2-171e-4aab-8b6b-6c4a4aaf0b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.585490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409de2-171e-4aab-8b6b-6c4a4aaf0b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.586164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da409de2-171e-4aab-8b6b-6c4a4aaf0b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:59.589131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da409de2-171e-4aab-8b6b-6c4a4aaf0b64 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:59.609642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 656/1297] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:37:59.611125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da409de2-171e-4aab-8b6b-6c4a4aaf0b64 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:59.625993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da409de2-171e-4aab-8b6b-6c4a4aaf0b64 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:59.640799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b833f2-6f18-43c9-86d5-09cc5d4d7cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:59.647575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b833f2-6f18-43c9-86d5-09cc5d4d7cee admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.647861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b833f2-6f18-43c9-86d5-09cc5d4d7cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.652870 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b833f2-6f18-43c9-86d5-09cc5d4d7cee admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.653572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b833f2-6f18-43c9-86d5-09cc5d4d7cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:59.655894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 642/1298] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Oct 12 12:37:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:37:59.656548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2108f57-ee51-4e94-b42f-540334b3808b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:59.657916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b833f2-6f18-43c9-86d5-09cc5d4d7cee admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:59.668091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2108f57-ee51-4e94-b42f-540334b3808b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:59.682293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949484c-a9ab-47bd-900d-d631ffa37401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:59.682293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949484c-a9ab-47bd-900d-d631ffa37401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.682293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949484c-a9ab-47bd-900d-d631ffa37401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.682293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949484c-a9ab-47bd-900d-d631ffa37401 admin admin] PATH_INFO: `/v3/projects/fd0b0947b9c44543bb127da0ec4f3ecf/users/2f96df1d3da246c7937dd44749967717/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.691576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949484c-a9ab-47bd-900d-d631ffa37401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0b0947b9c44543bb127da0ec4f3ecf, user_id=2f96df1d3da246c7937dd44749967717, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:59.694314 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949484c-a9ab-47bd-900d-d631ffa37401 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:59.771416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:59.782530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 657/1299] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:37:59 2022] POST /identity/v3/projects => generated 363 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:37:59.783476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b833f2-6f18-43c9-86d5-09cc5d4d7cee admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:59.833713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 643/1300] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:37:59 2022] PUT /identity/v3/projects/fd0b0947b9c44543bb127da0ec4f3ecf/users/2f96df1d3da246c7937dd44749967717/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:59.834929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949484c-a9ab-47bd-900d-d631ffa37401 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:59.880983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:59.889769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b833f2-6f18-43c9-86d5-09cc5d4d7cee admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:59.913960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949484c-a9ab-47bd-900d-d631ffa37401 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:59.918952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e724964-55af-4da0-9ec4-f5b77178bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:59.919939 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724964-55af-4da0-9ec4-f5b77178bb7c admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.919939 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724964-55af-4da0-9ec4-f5b77178bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.920274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724964-55af-4da0-9ec4-f5b77178bb7c admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:59.920917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e724964-55af-4da0-9ec4-f5b77178bb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:59.922886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e724964-55af-4da0-9ec4-f5b77178bb7c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:59.966191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38e412-f1c8-415d-8216-4efdc71cef6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:59.966191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e412-f1c8-415d-8216-4efdc71cef6b service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:59.966191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e412-f1c8-415d-8216-4efdc71cef6b service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:59.966191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e412-f1c8-415d-8216-4efdc71cef6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.022231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 644/1301] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:37:59 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:00.023861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e724964-55af-4da0-9ec4-f5b77178bb7c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:00.041259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e724964-55af-4da0-9ec4-f5b77178bb7c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:00.050685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bd6b6-e2e1-459f-a527-2a829db9db60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:00.051840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bd6b6-e2e1-459f-a527-2a829db9db60 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.052355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bd6b6-e2e1-459f-a527-2a829db9db60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.052933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bd6b6-e2e1-459f-a527-2a829db9db60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.054040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bd6b6-e2e1-459f-a527-2a829db9db60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.059717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bd6b6-e2e1-459f-a527-2a829db9db60 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.061812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38e412-f1c8-415d-8216-4efdc71cef6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.064254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38e412-f1c8-415d-8216-4efdc71cef6b service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.077539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 645/1302] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:00.078312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 658/1303] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:37:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:00.079113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38e412-f1c8-415d-8216-4efdc71cef6b service nova] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:00.081842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bd6b6-e2e1-459f-a527-2a829db9db60 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:00.091875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38e412-f1c8-415d-8216-4efdc71cef6b service nova] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:00.098933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca813c-3df3-4e69-913a-ddd7b7b71ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:00.099776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca813c-3df3-4e69-913a-ddd7b7b71ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.100052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca813c-3df3-4e69-913a-ddd7b7b71ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.100364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca813c-3df3-4e69-913a-ddd7b7b71ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.101023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca813c-3df3-4e69-913a-ddd7b7b71ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.102139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bd6b6-e2e1-459f-a527-2a829db9db60 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:00.102862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca813c-3df3-4e69-913a-ddd7b7b71ccb admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.108617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd5784-423f-4126-a643-895883180a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:00.109683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd5784-423f-4126-a643-895883180a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.110342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd5784-423f-4126-a643-895883180a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.110867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd5784-423f-4126-a643-895883180a18 admin admin] PATH_INFO: `/v3/projects/fd0b0947b9c44543bb127da0ec4f3ecf/users/2f96df1d3da246c7937dd44749967717/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.113051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd5784-423f-4126-a643-895883180a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0b0947b9c44543bb127da0ec4f3ecf, user_id=2f96df1d3da246c7937dd44749967717, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.116514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd5784-423f-4126-a643-895883180a18 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.119672 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 659/1304] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:00.132305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca813c-3df3-4e69-913a-ddd7b7b71ccb admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:00.134882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 646/1305] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:00 2022] PUT /identity/v3/projects/fd0b0947b9c44543bb127da0ec4f3ecf/users/2f96df1d3da246c7937dd44749967717/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:00.137017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca813c-3df3-4e69-913a-ddd7b7b71ccb admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:00.146104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3ede-4089-49e8-ad4f-36c00c6d271e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.146695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3ede-4089-49e8-ad4f-36c00c6d271e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.147100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da3ede-4089-49e8-ad4f-36c00c6d271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.215994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-26da3ede-4089-49e8-ad4f-36c00c6d271e None None] MFA Rules not processed for user `2f96df1d3da246c7937dd44749967717`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:00.217718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-26da3ede-4089-49e8-ad4f-36c00c6d271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:00.236973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5d73b-9276-4364-966b-8e08aea8c53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:00.237652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5d73b-9276-4364-966b-8e08aea8c53c admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.237946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5d73b-9276-4364-966b-8e08aea8c53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.238286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5d73b-9276-4364-966b-8e08aea8c53c admin admin] PATH_INFO: `/v3/projects/b35637e82dbb44feaeb318f61075a386/users/cc0a0200fd5a4de18f0d51283b45a750/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.263695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5d73b-9276-4364-966b-8e08aea8c53c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35637e82dbb44feaeb318f61075a386, user_id=cc0a0200fd5a4de18f0d51283b45a750, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.268178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5d73b-9276-4364-966b-8e08aea8c53c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.374704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 647/1306] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:00.377848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 660/1307] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:00 2022] PUT /identity/v3/projects/b35637e82dbb44feaeb318f61075a386/users/cc0a0200fd5a4de18f0d51283b45a750/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:00.383160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da3ede-4089-49e8-ad4f-36c00c6d271e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:00.385911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5d73b-9276-4364-966b-8e08aea8c53c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:00.400488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da3ede-4089-49e8-ad4f-36c00c6d271e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:00.543697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069dec8-dc1a-4e7c-977d-058b317e7d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:00.546400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069dec8-dc1a-4e7c-977d-058b317e7d97 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.546888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069dec8-dc1a-4e7c-977d-058b317e7d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.547478 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069dec8-dc1a-4e7c-977d-058b317e7d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.549671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069dec8-dc1a-4e7c-977d-058b317e7d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.553469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5d73b-9276-4364-966b-8e08aea8c53c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:00.555479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069dec8-dc1a-4e7c-977d-058b317e7d97 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.565203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01caf02-686d-4ebc-8bf2-a25877991025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:00.566476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01caf02-686d-4ebc-8bf2-a25877991025 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.566765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01caf02-686d-4ebc-8bf2-a25877991025 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.567144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01caf02-686d-4ebc-8bf2-a25877991025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.587610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 648/1308] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:00.588486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98839391-1a01-4866-af51-5e9169abdddb None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.588779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98839391-1a01-4866-af51-5e9169abdddb None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.589056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98839391-1a01-4866-af51-5e9169abdddb None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.593683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 649/1309] 10.222.0.44 () {54 vars in 912 bytes} [Wed Oct 12 12:38:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:38:00.595561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98839391-1a01-4866-af51-5e9169abdddb None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:00.619233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98839391-1a01-4866-af51-5e9169abdddb None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:00.625692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b942991-68cc-4a7e-b386-3c3d9bfb8237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:00.626625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b942991-68cc-4a7e-b386-3c3d9bfb8237 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.627088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b942991-68cc-4a7e-b386-3c3d9bfb8237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.627714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b942991-68cc-4a7e-b386-3c3d9bfb8237 admin admin] PATH_INFO: `/v3/projects/b35637e82dbb44feaeb318f61075a386/users/cc0a0200fd5a4de18f0d51283b45a750/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.631022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b942991-68cc-4a7e-b386-3c3d9bfb8237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35637e82dbb44feaeb318f61075a386, user_id=cc0a0200fd5a4de18f0d51283b45a750, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.634396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b942991-68cc-4a7e-b386-3c3d9bfb8237 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.656908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 650/1310] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:00 2022] PUT /identity/v3/projects/b35637e82dbb44feaeb318f61075a386/users/cc0a0200fd5a4de18f0d51283b45a750/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:00.660657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a9378-9a47-4580-acad-0046d6350ae3 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.660657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a9378-9a47-4580-acad-0046d6350ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.660657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a9378-9a47-4580-acad-0046d6350ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.681036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:00.690004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6f0a9378-9a47-4580-acad-0046d6350ae3 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:00.691666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6f0a9378-9a47-4580-acad-0046d6350ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:00.728575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01caf02-686d-4ebc-8bf2-a25877991025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.728575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01caf02-686d-4ebc-8bf2-a25877991025 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.753613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 661/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:00.754140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70396bc-d1e1-476f-bc0c-3989443bd72a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.754140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70396bc-d1e1-476f-bc0c-3989443bd72a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.754854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70396bc-d1e1-476f-bc0c-3989443bd72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.773883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:00.803347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 651/1312] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:38:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:00.815143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d70396bc-d1e1-476f-bc0c-3989443bd72a None None] MFA Rules not processed for user `cc0a0200fd5a4de18f0d51283b45a750`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:00.816778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d70396bc-d1e1-476f-bc0c-3989443bd72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:00.823312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a9378-9a47-4580-acad-0046d6350ae3 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:00.936191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a9378-9a47-4580-acad-0046d6350ae3 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:00.940833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac568a95-85e8-4114-9e77-2fe192c079f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:00.942434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac568a95-85e8-4114-9e77-2fe192c079f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:00.942434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac568a95-85e8-4114-9e77-2fe192c079f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:00.944236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac568a95-85e8-4114-9e77-2fe192c079f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:00.949994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 662/1313] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:00.962572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac568a95-85e8-4114-9e77-2fe192c079f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:00.964283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac568a95-85e8-4114-9e77-2fe192c079f4 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:00.979818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 652/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:01.027916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70396bc-d1e1-476f-bc0c-3989443bd72a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:01.038196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70396bc-d1e1-476f-bc0c-3989443bd72a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:01.046638 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bcb00-6fa3-4943-89b4-519bfa048481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:01.047685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bcb00-6fa3-4943-89b4-519bfa048481 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:01.048490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bcb00-6fa3-4943-89b4-519bfa048481 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:01.049084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bcb00-6fa3-4943-89b4-519bfa048481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:01.065763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bcb00-6fa3-4943-89b4-519bfa048481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:01.068233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bcb00-6fa3-4943-89b4-519bfa048481 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:01.094045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 663/1315] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:01.132403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:01.230526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac568a95-85e8-4114-9e77-2fe192c079f4 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:01.240942 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac568a95-85e8-4114-9e77-2fe192c079f4 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:01.245406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b134303-c8aa-4f4b-8fbb-9eb8a52505d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:01.245406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b134303-c8aa-4f4b-8fbb-9eb8a52505d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:01.245720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b134303-c8aa-4f4b-8fbb-9eb8a52505d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:01.246527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b134303-c8aa-4f4b-8fbb-9eb8a52505d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:01.348501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b134303-c8aa-4f4b-8fbb-9eb8a52505d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:01.349631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b134303-c8aa-4f4b-8fbb-9eb8a52505d5 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:01.371009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 653/1316] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:01.937230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:02.204176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bcb00-6fa3-4943-89b4-519bfa048481 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:02.215434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bcb00-6fa3-4943-89b4-519bfa048481 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:02.223956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe01258-4876-4c3f-aad6-74ea2698de2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:02.224773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01258-4876-4c3f-aad6-74ea2698de2a service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:02.225063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01258-4876-4c3f-aad6-74ea2698de2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:02.225335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe01258-4876-4c3f-aad6-74ea2698de2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:02.358893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe01258-4876-4c3f-aad6-74ea2698de2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:02.360717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe01258-4876-4c3f-aad6-74ea2698de2a service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:02.394536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 664/1317] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:02.563483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab563-eb4d-42b0-94fe-2021a817a962 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:02.565310 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab563-eb4d-42b0-94fe-2021a817a962 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:02.565693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab563-eb4d-42b0-94fe-2021a817a962 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:02.567385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 654/1318] 10.222.0.44 () {54 vars in 961 bytes} [Wed Oct 12 12:38:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:38:02.581094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f3c0e-2c6f-4bb7-a8ed-b65ea1a8b068 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:02.581442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f3c0e-2c6f-4bb7-a8ed-b65ea1a8b068 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:02.581731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f3c0e-2c6f-4bb7-a8ed-b65ea1a8b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:02.621408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-806f3c0e-2c6f-4bb7-a8ed-b65ea1a8b068 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:02.623401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-806f3c0e-2c6f-4bb7-a8ed-b65ea1a8b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:02.635543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 665/1319] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Oct 12 12:38:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:02.641451 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea19d02-1a06-483f-8eb6-47b9260b2812 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:02.641749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea19d02-1a06-483f-8eb6-47b9260b2812 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:02.642548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea19d02-1a06-483f-8eb6-47b9260b2812 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:02.643816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 655/1320] 10.222.0.44 () {54 vars in 912 bytes} [Wed Oct 12 12:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:38:02.650623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9dcc-c419-40ad-a870-837707ff3af2 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:02.651523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9dcc-c419-40ad-a870-837707ff3af2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:02.653452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9dcc-c419-40ad-a870-837707ff3af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:02.682780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5d3b9dcc-c419-40ad-a870-837707ff3af2 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:02.684303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5d3b9dcc-c419-40ad-a870-837707ff3af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:02.704016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 666/1321] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:38:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:02.718213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea19d02-1a06-483f-8eb6-47b9260b2812 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:02.732876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea19d02-1a06-483f-8eb6-47b9260b2812 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:02.741585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ae1f2-6699-4e6d-b25d-5c06538baf4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:02.742374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae1f2-6699-4e6d-b25d-5c06538baf4f service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:02.742812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae1f2-6699-4e6d-b25d-5c06538baf4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:02.743159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ae1f2-6699-4e6d-b25d-5c06538baf4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:02.776998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ae1f2-6699-4e6d-b25d-5c06538baf4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:02.780417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ae1f2-6699-4e6d-b25d-5c06538baf4f service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:02.796853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 656/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:10.129853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d11e9-9edf-4613-9c04-ec7bfcdf7e9f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:10.130409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d11e9-9edf-4613-9c04-ec7bfcdf7e9f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:10.130409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d11e9-9edf-4613-9c04-ec7bfcdf7e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:10.198492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7d3d11e9-9edf-4613-9c04-ec7bfcdf7e9f None None] MFA Rules not processed for user `39543aedd1834f68a9d0a4ef174822c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:10.200289 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7d3d11e9-9edf-4613-9c04-ec7bfcdf7e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:10.358582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 667/1323] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:10.377950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ae1f2-6699-4e6d-b25d-5c06538baf4f service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:10.395726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d11e9-9edf-4613-9c04-ec7bfcdf7e9f None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:10.436511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d11e9-9edf-4613-9c04-ec7bfcdf7e9f None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:10.451031 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7922e-28b6-4a6e-a2b0-32ac99a56097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:10.451665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7922e-28b6-4a6e-a2b0-32ac99a56097 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:10.451944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7922e-28b6-4a6e-a2b0-32ac99a56097 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:10.452258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7922e-28b6-4a6e-a2b0-32ac99a56097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:10.487032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7922e-28b6-4a6e-a2b0-32ac99a56097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:10.499796 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7922e-28b6-4a6e-a2b0-32ac99a56097 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:10.557385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 668/1324] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:10.624576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ae1f2-6699-4e6d-b25d-5c06538baf4f service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:10.651218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:10.652086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:10.657550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:10.659463 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] PATH_INFO: `/v3/users/4eb284a8d97d42dfb361595571ad531c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:10.670138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb284a8d97d42dfb361595571ad531c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:10.673156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:10.880193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb284a8d97d42dfb361595571ad531c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:10.962291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb284a8d97d42dfb361595571ad531c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:11.040905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb284a8d97d42dfb361595571ad531c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:11.042069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb284a8d97d42dfb361595571ad531c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:11.058916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb284a8d97d42dfb361595571ad531c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:11.095961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 657/1325] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:10 2022] DELETE /identity/v3/users/4eb284a8d97d42dfb361595571ad531c => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:11.098738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9ea3-6acb-4966-88bc-032228cb7d98 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:11.099048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9ea3-6acb-4966-88bc-032228cb7d98 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:11.099314 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f9ea3-6acb-4966-88bc-032228cb7d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:11.141825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-8d6f9ea3-6acb-4966-88bc-032228cb7d98 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:11.143665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8d6f9ea3-6acb-4966-88bc-032228cb7d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:11.212622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:11.306297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 669/1326] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:11.413426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:11.415136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:11.421774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:11.525706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12be7d3-72c3-4729-a26c-d80b352c473f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:11.532106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cca3a-0524-4579-b8c1-04a388f7253b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:11.533073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cca3a-0524-4579-b8c1-04a388f7253b service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:11.533592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cca3a-0524-4579-b8c1-04a388f7253b service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:11.534230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cca3a-0524-4579-b8c1-04a388f7253b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:11.555047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cca3a-0524-4579-b8c1-04a388f7253b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:11.558374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cca3a-0524-4579-b8c1-04a388f7253b service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:11.572735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 658/1327] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:11.638553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b51dab-10fa-4cd2-9861-17498ca9bc58 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:11.639157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b51dab-10fa-4cd2-9861-17498ca9bc58 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:11.639624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b51dab-10fa-4cd2-9861-17498ca9bc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:11.698966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-98b51dab-10fa-4cd2-9861-17498ca9bc58 None None] MFA Rules not processed for user `df8da1fe608d4dd6acb6b7298ca56d45`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:11.701325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-98b51dab-10fa-4cd2-9861-17498ca9bc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:11.799654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c34757-fa32-4b18-8a05-b7c34b37babe None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:11.801182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c34757-fa32-4b18-8a05-b7c34b37babe None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:11.801753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c34757-fa32-4b18-8a05-b7c34b37babe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:11.850763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 670/1328] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:11.874543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e0c34757-fa32-4b18-8a05-b7c34b37babe None None] MFA Rules not processed for user `df838ad816744832baa283dabe540394`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:11.876566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e0c34757-fa32-4b18-8a05-b7c34b37babe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:11.877308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b51dab-10fa-4cd2-9861-17498ca9bc58 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:11.878854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-98b51dab-10fa-4cd2-9861-17498ca9bc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:12.070514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b51dab-10fa-4cd2-9861-17498ca9bc58 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:12.070620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 659/1329] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:12.070620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6e3a2-5d01-4698-a6fd-c15ba41132b3 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:12.070620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6e3a2-5d01-4698-a6fd-c15ba41132b3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:12.070840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6e3a2-5d01-4698-a6fd-c15ba41132b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:12.076389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531354c-aa44-40cf-a3b9-3e75b64a2959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:12.077099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531354c-aa44-40cf-a3b9-3e75b64a2959 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:12.077493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531354c-aa44-40cf-a3b9-3e75b64a2959 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:12.078176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531354c-aa44-40cf-a3b9-3e75b64a2959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:12.093366 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531354c-aa44-40cf-a3b9-3e75b64a2959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:12.095083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531354c-aa44-40cf-a3b9-3e75b64a2959 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:12.122281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-caa6e3a2-5d01-4698-a6fd-c15ba41132b3 None None] MFA Rules not processed for user `df838ad816744832baa283dabe540394`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:12.125026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 671/1330] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:12.141810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-caa6e3a2-5d01-4698-a6fd-c15ba41132b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:12.165693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 660/1331] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:12.362884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531354c-aa44-40cf-a3b9-3e75b64a2959 service nova] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:12.390335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531354c-aa44-40cf-a3b9-3e75b64a2959 service nova] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:12.412030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0056a1-cc74-4d38-9f4f-cd165535ad40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:12.413377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0056a1-cc74-4d38-9f4f-cd165535ad40 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:12.414032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0056a1-cc74-4d38-9f4f-cd165535ad40 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:12.414453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0056a1-cc74-4d38-9f4f-cd165535ad40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:12.429850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0056a1-cc74-4d38-9f4f-cd165535ad40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:12.431520 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0056a1-cc74-4d38-9f4f-cd165535ad40 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:12.444700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 672/1332] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:13.250797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6e3a2-5d01-4698-a6fd-c15ba41132b3 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:13.252869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-caa6e3a2-5d01-4698-a6fd-c15ba41132b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:13.289160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6e3a2-5d01-4698-a6fd-c15ba41132b3 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:13.303647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:13.304995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:13.305446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:13.306165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] PATH_INFO: `/v3/projects/08492244b10d4219a38e24a648a90c56` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:13.315016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08492244b10d4219a38e24a648a90c56)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:13.318604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: WARNING py.warnings [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:38:13.318604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: warnings.warn(msg) Oct 12 12:38:13.318604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: Oct 12 12:38:13.325833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:13.425321 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46638f-4836-42ba-94de-0333f96637e1 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:13.426637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46638f-4836-42ba-94de-0333f96637e1 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:13.427179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46638f-4836-42ba-94de-0333f96637e1 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:13.429448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 673/1333] 10.222.0.44 () {54 vars in 912 bytes} [Wed Oct 12 12:38:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:38:13.436369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a417d-8fc3-4d4c-92c0-0c32039e0eb5 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:13.436878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a417d-8fc3-4d4c-92c0-0c32039e0eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:13.437318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a417d-8fc3-4d4c-92c0-0c32039e0eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:13.531065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-171a417d-8fc3-4d4c-92c0-0c32039e0eb5 None None] MFA Rules not processed for user `db04575738b04f2d95803498e19554c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:13.532757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-171a417d-8fc3-4d4c-92c0-0c32039e0eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:13.577527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08492244b10d4219a38e24a648a90c56', 'request_id': 'req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:13.584213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:13.603199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08492244b10d4219a38e24a648a90c56', 'request_id': 'req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:13.651363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08492244b10d4219a38e24a648a90c56', 'request_id': 'req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:13.724916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 674/1334] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:13.727094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3005c0c-6b11-4641-995f-9a27d89236f8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:13.727738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3005c0c-6b11-4641-995f-9a27d89236f8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:13.728728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3005c0c-6b11-4641-995f-9a27d89236f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:13.754243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] The token cache is being invalidate because project 08492244b10d4219a38e24a648a90c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:13.754243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:13.754243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:13.755262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 661/1335] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:13 2022] DELETE /identity/v3/projects/08492244b10d4219a38e24a648a90c56 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:13.756057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:13.762920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:13.786019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-f3005c0c-6b11-4641-995f-9a27d89236f8 None None] MFA Rules not processed for user `19d863a45ac14c50b940fbc217b70929`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:13.787258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f3005c0c-6b11-4641-995f-9a27d89236f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:13.794111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7bfdb-ba6e-4720-b2c8-30bf27396b70 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:13.803448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:13.804099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:13.804383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:13.804658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:13.811166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:13.859723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:13.876724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:13.905896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:13.945195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 662/1336] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:13.952414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:13.960628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:14.208828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 675/1337] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:14.211722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169acaae-2c56-47f7-a51b-5cd778f66adc None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:14.212255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169acaae-2c56-47f7-a51b-5cd778f66adc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:14.212750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169acaae-2c56-47f7-a51b-5cd778f66adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:14.235432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-169acaae-2c56-47f7-a51b-5cd778f66adc None None] MFA Rules not processed for user `19d863a45ac14c50b940fbc217b70929`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:14.237895 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-169acaae-2c56-47f7-a51b-5cd778f66adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:14.249312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 676/1338] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:14.260147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89dcb-e8b2-4a36-b305-2b32479dc941 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:14.266661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:14.271793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169acaae-2c56-47f7-a51b-5cd778f66adc None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:14.273229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-169acaae-2c56-47f7-a51b-5cd778f66adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:14.274375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:14.274661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:14.274950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] PATH_INFO: `/v3/users/2f96df1d3da246c7937dd44749967717` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:14.276558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f96df1d3da246c7937dd44749967717)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:14.279616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:14.468537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169acaae-2c56-47f7-a51b-5cd778f66adc None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:14.481371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:14.483373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:14.484760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:14.487460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:14.518956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:14.520836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:14.543566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 677/1339] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:14.597266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f96df1d3da246c7937dd44749967717'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:14.637973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f96df1d3da246c7937dd44749967717'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:14.755743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f96df1d3da246c7937dd44749967717'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:14.756408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f96df1d3da246c7937dd44749967717'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:14.768819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-934755fe-b929-4f09-8e39-e80ab3abb523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f96df1d3da246c7937dd44749967717'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:14.779295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 663/1340] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:13 2022] DELETE /identity/v3/users/2f96df1d3da246c7937dd44749967717 => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:15.654718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:15.664746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:15.755729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:15.814947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff17407-233d-4a10-8159-c1940c2fd7eb service nova] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:15.924852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:15.925713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:15.926312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:15.927363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:15.928601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:15.952381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9aba-e6e9-4cb7-8b2e-0b0b58e8262c None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:15.952931 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9aba-e6e9-4cb7-8b2e-0b0b58e8262c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:15.953069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9aba-e6e9-4cb7-8b2e-0b0b58e8262c None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:15.955140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 664/1341] 10.222.0.44 () {54 vars in 912 bytes} [Wed Oct 12 12:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:38:15.964783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9378-ab40-4515-93ed-cfb6fdd796ed None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:15.971089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9378-ab40-4515-93ed-cfb6fdd796ed None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:15.971089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e9378-ab40-4515-93ed-cfb6fdd796ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:16.006920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:16.008715 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:16.044341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1b5e9378-ab40-4515-93ed-cfb6fdd796ed None None] MFA Rules not processed for user `db04575738b04f2d95803498e19554c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:16.048239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1b5e9378-ab40-4515-93ed-cfb6fdd796ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:16.079007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 678/1342] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:16.080027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:16.084840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:16.214464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 665/1343] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:38:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:16.304134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641a6de-108a-45b2-a4a9-3f1be23f1e96 service placement] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:16.307566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e9378-ab40-4515-93ed-cfb6fdd796ed None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:16.309477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1b5e9378-ab40-4515-93ed-cfb6fdd796ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:16.317674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:16.321969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:16.321969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:16.321969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] PATH_INFO: `/v3/users/b65fe96949fe4f87970dc5657b2125f5` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:16.328142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65fe96949fe4f87970dc5657b2125f5)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:16.328142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: WARNING py.warnings [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:38:16.328142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: warnings.warn(msg) Oct 12 12:38:16.328142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: Oct 12 12:38:16.328142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:16.332080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e9378-ab40-4515-93ed-cfb6fdd796ed None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:16.347783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:16.348503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:16.348971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:16.349399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:16.374880 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:16.378798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:16.403132 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 666/1344] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:16.432294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:16.432294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:16.461562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccd5af-0f82-4be5-a822-c3117b3edd5e service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:16.474894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:16.475549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:16.476848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:16.478077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] PATH_INFO: `/v3/projects/fd0b0947b9c44543bb127da0ec4f3ecf` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:16.479688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0b0947b9c44543bb127da0ec4f3ecf)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:16.487680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:16.506620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:16.730225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0b0947b9c44543bb127da0ec4f3ecf', 'request_id': 'req-018ed040-e8c6-48e9-9aec-2365f7489018'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.738301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0b0947b9c44543bb127da0ec4f3ecf', 'request_id': 'req-018ed040-e8c6-48e9-9aec-2365f7489018'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.746454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0b0947b9c44543bb127da0ec4f3ecf', 'request_id': 'req-018ed040-e8c6-48e9-9aec-2365f7489018'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.749959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65fe96949fe4f87970dc5657b2125f5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.787102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65fe96949fe4f87970dc5657b2125f5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.788404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65fe96949fe4f87970dc5657b2125f5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.796498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:16.818933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] The token cache is being invalidate because project fd0b0947b9c44543bb127da0ec4f3ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:16.819729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.821098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-018ed040-e8c6-48e9-9aec-2365f7489018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.823833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 667/1345] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:16 2022] DELETE /identity/v3/projects/fd0b0947b9c44543bb127da0ec4f3ecf => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:16.826232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a3567-d86b-4196-82da-551899830862 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:16.830773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a3567-d86b-4196-82da-551899830862 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:16.830773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a3567-d86b-4196-82da-551899830862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:16.874544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65fe96949fe4f87970dc5657b2125f5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.883029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a55a3567-d86b-4196-82da-551899830862 None None] MFA Rules not processed for user `39543aedd1834f68a9d0a4ef174822c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:16.884589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a55a3567-d86b-4196-82da-551899830862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:16.887438 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5ce762ce-a403-4a49-9539-c9c07680caff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65fe96949fe4f87970dc5657b2125f5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:16.903837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 679/1346] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:16 2022] DELETE /identity/v3/users/b65fe96949fe4f87970dc5657b2125f5 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:16.907145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340e00-249e-4ede-a950-c988ab974f88 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:16.907669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340e00-249e-4ede-a950-c988ab974f88 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:16.909431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340e00-249e-4ede-a950-c988ab974f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:16.948595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-45340e00-249e-4ede-a950-c988ab974f88 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:16.951919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-45340e00-249e-4ede-a950-c988ab974f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:17.045764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 668/1347] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:17.048221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9fcb-79c4-47d5-9ffa-1922cc18fd9d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:17.049795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9fcb-79c4-47d5-9ffa-1922cc18fd9d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:17.050135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9fcb-79c4-47d5-9ffa-1922cc18fd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:17.119623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d9ed9fcb-79c4-47d5-9ffa-1922cc18fd9d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:17.120856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d9ed9fcb-79c4-47d5-9ffa-1922cc18fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:17.299575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 669/1348] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:17.300974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed9fcb-79c4-47d5-9ffa-1922cc18fd9d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:17.308591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d9ed9fcb-79c4-47d5-9ffa-1922cc18fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:17.519542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 680/1349] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 615 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:17.541690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724832f8-86ce-4c06-a1a0-7c5f5250f15b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:17.541998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724832f8-86ce-4c06-a1a0-7c5f5250f15b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:17.542282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724832f8-86ce-4c06-a1a0-7c5f5250f15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:17.551330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed9fcb-79c4-47d5-9ffa-1922cc18fd9d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:17.570008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39392b02-f7f9-4067-ae8c-79722ec443cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:17.570365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-724832f8-86ce-4c06-a1a0-7c5f5250f15b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:17.570647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392b02-f7f9-4067-ae8c-79722ec443cf service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:17.570929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392b02-f7f9-4067-ae8c-79722ec443cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:17.571204 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392b02-f7f9-4067-ae8c-79722ec443cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:17.578468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-724832f8-86ce-4c06-a1a0-7c5f5250f15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:17.601146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39392b02-f7f9-4067-ae8c-79722ec443cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:17.602648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39392b02-f7f9-4067-ae8c-79722ec443cf service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:17.625380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 681/1350] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:17.639240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724832f8-86ce-4c06-a1a0-7c5f5250f15b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:17.650423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 670/1351] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:17.666622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724832f8-86ce-4c06-a1a0-7c5f5250f15b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:17.675797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:17.676666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:17.677402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:17.678137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:17.679424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:17.680841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:17.690484 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:17.730141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 682/1352] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:38:17.730758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39392b02-f7f9-4067-ae8c-79722ec443cf service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:17.751924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39392b02-f7f9-4067-ae8c-79722ec443cf service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:17.773424 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776c9bb-cfa1-474f-9f69-b64f8db65742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:17.774083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776c9bb-cfa1-474f-9f69-b64f8db65742 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:17.774364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776c9bb-cfa1-474f-9f69-b64f8db65742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:17.775011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776c9bb-cfa1-474f-9f69-b64f8db65742 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:17.775667 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776c9bb-cfa1-474f-9f69-b64f8db65742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:17.789793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776c9bb-cfa1-474f-9f69-b64f8db65742 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:17.954582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 671/1353] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:17 2022] POST /identity/v3/projects => generated 399 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:17.960825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:17.984368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0ba09-5d03-4c80-a13a-c6fb262d6877 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:18.044177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70125d3-aca2-4c56-b846-e8a3bf4fca31 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.044468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70125d3-aca2-4c56-b846-e8a3bf4fca31 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.044743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70125d3-aca2-4c56-b846-e8a3bf4fca31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.093388 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f70125d3-aca2-4c56-b846-e8a3bf4fca31 None None] MFA Rules not processed for user `19d863a45ac14c50b940fbc217b70929`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:18.094649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f70125d3-aca2-4c56-b846-e8a3bf4fca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:18.192384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd19d-96cc-4a22-aefb-aa20faaa8edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:18.193379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd19d-96cc-4a22-aefb-aa20faaa8edc admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.193669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd19d-96cc-4a22-aefb-aa20faaa8edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.193958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd19d-96cc-4a22-aefb-aa20faaa8edc admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.194648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dd19d-96cc-4a22-aefb-aa20faaa8edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:18.198714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dd19d-96cc-4a22-aefb-aa20faaa8edc admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:18.268304 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 672/1354] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:18.269356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70125d3-aca2-4c56-b846-e8a3bf4fca31 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:18.270950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f70125d3-aca2-4c56-b846-e8a3bf4fca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:18.297741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70125d3-aca2-4c56-b846-e8a3bf4fca31 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:18.305760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:18.306655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.307007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.307411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] PATH_INFO: `/v3/users/8056f7de10564ccf9bdfcd035b721532` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.309646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8056f7de10564ccf9bdfcd035b721532)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:18.318492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:18.330459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 683/1355] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:17 2022] POST /identity/v3/users => generated 300 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:18.336913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd19d-96cc-4a22-aefb-aa20faaa8edc admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:18.358018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd19d-96cc-4a22-aefb-aa20faaa8edc admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:18.383570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b09e87-7f3c-4f55-b090-3fc5940d0b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:18.384211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b09e87-7f3c-4f55-b090-3fc5940d0b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.384531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b09e87-7f3c-4f55-b090-3fc5940d0b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.384838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b09e87-7f3c-4f55-b090-3fc5940d0b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.385387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b09e87-7f3c-4f55-b090-3fc5940d0b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:18.397322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b09e87-7f3c-4f55-b090-3fc5940d0b05 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:18.427405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 684/1356] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:18.431204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b09e87-7f3c-4f55-b090-3fc5940d0b05 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:18.447164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b09e87-7f3c-4f55-b090-3fc5940d0b05 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:18.453445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751c69a-92a8-4dbe-92ab-0b8e8009a08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:18.459136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c69a-92a8-4dbe-92ab-0b8e8009a08f admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.459136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c69a-92a8-4dbe-92ab-0b8e8009a08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.459136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c69a-92a8-4dbe-92ab-0b8e8009a08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.459136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751c69a-92a8-4dbe-92ab-0b8e8009a08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:18.459136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751c69a-92a8-4dbe-92ab-0b8e8009a08f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:18.472633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 685/1357] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:18.483137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751c69a-92a8-4dbe-92ab-0b8e8009a08f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:18.500560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751c69a-92a8-4dbe-92ab-0b8e8009a08f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:18.512295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d9d9f-ba0a-434a-9211-781de685d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:18.513629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d9d9f-ba0a-434a-9211-781de685d923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.514162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d9d9f-ba0a-434a-9211-781de685d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.514707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d9d9f-ba0a-434a-9211-781de685d923 admin admin] PATH_INFO: `/v3/projects/eb6288104d594741b1270fac948beae9/users/156ae60d9b5e47f9aab9b02c180fb642/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.550364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d9d9f-ba0a-434a-9211-781de685d923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6288104d594741b1270fac948beae9, user_id=156ae60d9b5e47f9aab9b02c180fb642, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:18.555020 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d9d9f-ba0a-434a-9211-781de685d923 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:18.629267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8056f7de10564ccf9bdfcd035b721532'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:18.650251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8056f7de10564ccf9bdfcd035b721532'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:18.705787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 686/1358] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:18 2022] PUT /identity/v3/projects/eb6288104d594741b1270fac948beae9/users/156ae60d9b5e47f9aab9b02c180fb642/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:18.712235 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4f4ff-a6a1-459c-acf2-e1418c32fe66 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.712772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4f4ff-a6a1-459c-acf2-e1418c32fe66 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.713232 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4f4ff-a6a1-459c-acf2-e1418c32fe66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.762834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b1e4f4ff-a6a1-459c-acf2-e1418c32fe66 None None] MFA Rules not processed for user `df8da1fe608d4dd6acb6b7298ca56d45`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:18.764403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b1e4f4ff-a6a1-459c-acf2-e1418c32fe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:18.822751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8056f7de10564ccf9bdfcd035b721532'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:18.824175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8056f7de10564ccf9bdfcd035b721532'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:18.886511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f8ec3b71-3ce8-4d75-954e-c11ce9a5b475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8056f7de10564ccf9bdfcd035b721532'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:18.911224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 673/1359] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:18 2022] DELETE /identity/v3/users/8056f7de10564ccf9bdfcd035b721532 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:18.914088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c9138-3508-415f-802a-80be6a3c651e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.914384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c9138-3508-415f-802a-80be6a3c651e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.914765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c9138-3508-415f-802a-80be6a3c651e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.939471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 687/1360] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:18.941059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21445-8e47-4c50-903d-d31aba3f79ac None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:18.941587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21445-8e47-4c50-903d-d31aba3f79ac None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:18.942195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21445-8e47-4c50-903d-d31aba3f79ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:18.968468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c01c9138-3508-415f-802a-80be6a3c651e None None] MFA Rules not processed for user `156ae60d9b5e47f9aab9b02c180fb642`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:18.970200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c01c9138-3508-415f-802a-80be6a3c651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:18.990305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e1e21445-8e47-4c50-903d-d31aba3f79ac None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:18.992008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e1e21445-8e47-4c50-903d-d31aba3f79ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:19.230981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 674/1361] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:19.232812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c9138-3508-415f-802a-80be6a3c651e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:19.235128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 688/1362] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:19.236168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21445-8e47-4c50-903d-d31aba3f79ac None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:19.241854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e1e21445-8e47-4c50-903d-d31aba3f79ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:19.245700 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c01c9138-3508-415f-802a-80be6a3c651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:19.266538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c9138-3508-415f-802a-80be6a3c651e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:19.272732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:19.273377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:19.273655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:19.274131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] PATH_INFO: `/v3/projects/2a372db0342f425cbd7dedccf68e7fd4` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:19.275720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a372db0342f425cbd7dedccf68e7fd4)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:19.284714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21445-8e47-4c50-903d-d31aba3f79ac None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:19.286451 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:19.289765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:19.298262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:19.299507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:19.299859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:19.300205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:19.300921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:19.302947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:19.437729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 689/1363] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:19 2022] POST /identity/v3/projects => generated 399 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:19.443651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:19.448629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:19.491552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a372db0342f425cbd7dedccf68e7fd4', 'request_id': 'req-73d1ca59-8374-4ea1-b61d-5bed534f3363'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:19.499829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a372db0342f425cbd7dedccf68e7fd4', 'request_id': 'req-73d1ca59-8374-4ea1-b61d-5bed534f3363'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:19.522993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a372db0342f425cbd7dedccf68e7fd4', 'request_id': 'req-73d1ca59-8374-4ea1-b61d-5bed534f3363'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:19.575464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:19.618795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] The token cache is being invalidate because project 2a372db0342f425cbd7dedccf68e7fd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:19.619258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:19.620067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:19.622692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 675/1364] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:19 2022] DELETE /identity/v3/projects/2a372db0342f425cbd7dedccf68e7fd4 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:19.623498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:19.625730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:19.646747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeaab0-2b07-4cd0-9390-ad96e88a44e1 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:19.733603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1ca59-8374-4ea1-b61d-5bed534f3363 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:19.743885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d192aba-a83b-4f10-b2b1-99dae35b8046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:19.744842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d192aba-a83b-4f10-b2b1-99dae35b8046 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:19.745127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d192aba-a83b-4f10-b2b1-99dae35b8046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:19.745397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d192aba-a83b-4f10-b2b1-99dae35b8046 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:19.746846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d192aba-a83b-4f10-b2b1-99dae35b8046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:19.748275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d192aba-a83b-4f10-b2b1-99dae35b8046 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:19.765372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:19.766040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:19.766320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:19.766588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:19.768416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:19.783518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:19.785096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:19.808632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 690/1365] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:19.809655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:19.811422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:19.825479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc402d7c-0260-456c-9b88-09e7861e4200 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:19.833467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:19.834694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:19.836235 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:19.836774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] PATH_INFO: `/v3/users/d81df313cba34fdd9717fee325ec8b1e` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:19.842473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81df313cba34fdd9717fee325ec8b1e)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:19.844647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:19.882223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 676/1366] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:19 2022] POST /identity/v3/users => generated 306 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:19.883318 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1d4c-a6b8-4528-ace6-152882a47d43 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:19.883603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1d4c-a6b8-4528-ace6-152882a47d43 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:19.883922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b1d4c-a6b8-4528-ace6-152882a47d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:19.957320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e95b1d4c-a6b8-4528-ace6-152882a47d43 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:19.960841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e95b1d4c-a6b8-4528-ace6-152882a47d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:19.993994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 677/1367] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:19.996875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b1d4c-a6b8-4528-ace6-152882a47d43 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:20.036948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b1d4c-a6b8-4528-ace6-152882a47d43 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:20.045076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472997d0-70c1-4013-985c-7a084d6dd6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:20.045704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472997d0-70c1-4013-985c-7a084d6dd6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:20.046028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472997d0-70c1-4013-985c-7a084d6dd6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:20.046303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472997d0-70c1-4013-985c-7a084d6dd6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:20.046997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472997d0-70c1-4013-985c-7a084d6dd6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:20.050343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472997d0-70c1-4013-985c-7a084d6dd6f4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:20.097576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 678/1368] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:20.097576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55f56a-99fa-4e93-98c6-ed887f0c3856 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:20.097576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55f56a-99fa-4e93-98c6-ed887f0c3856 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:20.097576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55f56a-99fa-4e93-98c6-ed887f0c3856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:20.142292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-da55f56a-99fa-4e93-98c6-ed887f0c3856 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:20.145509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-da55f56a-99fa-4e93-98c6-ed887f0c3856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:20.197481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 679/1369] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:20.198353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55f56a-99fa-4e93-98c6-ed887f0c3856 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:20.227917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55f56a-99fa-4e93-98c6-ed887f0c3856 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:20.259072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:20.259781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:20.259873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:20.260142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] PATH_INFO: `/v3/projects/5b32af54526643ab9c9646fd3b61641c/users/7598284269a0410ba7872901531f8203/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:20.297043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81df313cba34fdd9717fee325ec8b1e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:20.379208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b32af54526643ab9c9646fd3b61641c, user_id=7598284269a0410ba7872901531f8203, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:20.383490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:20.386465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81df313cba34fdd9717fee325ec8b1e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:20.387203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81df313cba34fdd9717fee325ec8b1e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:20.461873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 680/1370] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:20 2022] PUT /identity/v3/projects/5b32af54526643ab9c9646fd3b61641c/users/7598284269a0410ba7872901531f8203/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:20.463058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:20.467011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:20.482642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81df313cba34fdd9717fee325ec8b1e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:20.499717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81df313cba34fdd9717fee325ec8b1e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:20.511725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 691/1371] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:19 2022] DELETE /identity/v3/users/d81df313cba34fdd9717fee325ec8b1e => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:20.513003 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:20.517181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:20.652227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff0232-4e1d-43f4-960d-4160c8953434 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:20.705027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230edd7-f408-433d-9ebb-371ede5e76bb admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:20.721566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e03a1-f7e3-497b-815a-1746d09c6820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:20.723709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e03a1-f7e3-497b-815a-1746d09c6820 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:20.723943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e03a1-f7e3-497b-815a-1746d09c6820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:20.724214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e03a1-f7e3-497b-815a-1746d09c6820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:20.725113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e03a1-f7e3-497b-815a-1746d09c6820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:20.726777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e03a1-f7e3-497b-815a-1746d09c6820 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:20.747845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 692/1372] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:20.749691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08feed-50c4-417d-b5c4-9ed8b59cc436 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:20.750203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08feed-50c4-417d-b5c4-9ed8b59cc436 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:20.760182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08feed-50c4-417d-b5c4-9ed8b59cc436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:20.779745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-bd08feed-50c4-417d-b5c4-9ed8b59cc436 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:20.781585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bd08feed-50c4-417d-b5c4-9ed8b59cc436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:20.800188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 693/1373] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:20.801123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:20.801755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:20.802126 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:20.802489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:20.802812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08feed-50c4-417d-b5c4-9ed8b59cc436 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:20.803035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:20.803338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:20.805229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:20.822779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08feed-50c4-417d-b5c4-9ed8b59cc436 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:20.831128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ee385-63d3-4cfa-a307-564307c3da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:20.832442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee385-63d3-4cfa-a307-564307c3da62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:20.833434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee385-63d3-4cfa-a307-564307c3da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:20.833887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee385-63d3-4cfa-a307-564307c3da62 admin admin] PATH_INFO: `/v3/projects/5b32af54526643ab9c9646fd3b61641c/users/7598284269a0410ba7872901531f8203/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:20.838550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ee385-63d3-4cfa-a307-564307c3da62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b32af54526643ab9c9646fd3b61641c, user_id=7598284269a0410ba7872901531f8203, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:20.843800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ee385-63d3-4cfa-a307-564307c3da62 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:20.845196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 681/1374] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:38:20.849541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:20.854056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:20.859158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 694/1375] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:20 2022] PUT /identity/v3/projects/5b32af54526643ab9c9646fd3b61641c/users/7598284269a0410ba7872901531f8203/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:20.860547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ee385-63d3-4cfa-a307-564307c3da62 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:20.971464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242126-f610-45d7-8a75-46073701f772 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:21.128509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be46d4-0952-4e9b-a02a-8d13527e3996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:21.130066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be46d4-0952-4e9b-a02a-8d13527e3996 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:21.130367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be46d4-0952-4e9b-a02a-8d13527e3996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:21.130649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be46d4-0952-4e9b-a02a-8d13527e3996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:21.155766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be46d4-0952-4e9b-a02a-8d13527e3996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:21.157228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be46d4-0952-4e9b-a02a-8d13527e3996 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:21.170180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ee385-63d3-4cfa-a307-564307c3da62 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:21.191173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088824e-086e-4a74-88c8-482141444fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:21.192221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088824e-086e-4a74-88c8-482141444fda admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:21.192599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088824e-086e-4a74-88c8-482141444fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:21.192970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088824e-086e-4a74-88c8-482141444fda admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:21.193708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088824e-086e-4a74-88c8-482141444fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:21.194267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 682/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:21.195546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed59ca7-1905-4f6c-a9f4-f5179e4c77b1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:21.195831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed59ca7-1905-4f6c-a9f4-f5179e4c77b1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:21.196068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088824e-086e-4a74-88c8-482141444fda admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:21.200512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed59ca7-1905-4f6c-a9f4-f5179e4c77b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:21.306872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0ed59ca7-1905-4f6c-a9f4-f5179e4c77b1 None None] MFA Rules not processed for user `7598284269a0410ba7872901531f8203`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:21.313766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0ed59ca7-1905-4f6c-a9f4-f5179e4c77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:21.405665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 695/1377] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:20 2022] POST /identity/v3/projects => generated 355 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:21.409989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088824e-086e-4a74-88c8-482141444fda admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:21.520416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 683/1378] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:21 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:21.533424 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed59ca7-1905-4f6c-a9f4-f5179e4c77b1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:21.535845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0ed59ca7-1905-4f6c-a9f4-f5179e4c77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:21.651523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088824e-086e-4a74-88c8-482141444fda admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:21.658698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:21.663077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:21.663077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:21.663077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:21.663077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:21.665621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:21.693698 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed59ca7-1905-4f6c-a9f4-f5179e4c77b1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:21.817047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 696/1379] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:21 2022] POST /identity/v3/users => generated 278 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:21.818128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:21.822818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:21.843414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877cc69-e1fa-4709-a62e-fba5a3a457cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:21.844046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877cc69-e1fa-4709-a62e-fba5a3a457cd service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:21.846005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877cc69-e1fa-4709-a62e-fba5a3a457cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:21.846118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9deda-5480-48fe-b437-244b381b8338 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:21.846195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877cc69-e1fa-4709-a62e-fba5a3a457cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:21.852150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:21.853078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:21.854042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:21.855544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] PATH_INFO: `/v3/projects/d329d951b42740b88a3782cd08961ffb` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:21.857469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d329d951b42740b88a3782cd08961ffb)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:21.860956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:21.896358 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:22.027572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877cc69-e1fa-4709-a62e-fba5a3a457cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:22.029766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877cc69-e1fa-4709-a62e-fba5a3a457cd service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:22.053878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 684/1380] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:21 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:22.054798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877cc69-e1fa-4709-a62e-fba5a3a457cd service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:22.072811 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877cc69-e1fa-4709-a62e-fba5a3a457cd service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:22.097896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ecbc5-dc8d-4fce-bc62-9bb6dbe44e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:22.099015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ecbc5-dc8d-4fce-bc62-9bb6dbe44e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:22.100563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ecbc5-dc8d-4fce-bc62-9bb6dbe44e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:22.100858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ecbc5-dc8d-4fce-bc62-9bb6dbe44e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:22.103443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ecbc5-dc8d-4fce-bc62-9bb6dbe44e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:22.121136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ecbc5-dc8d-4fce-bc62-9bb6dbe44e43 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:22.128408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd329d951b42740b88a3782cd08961ffb', 'request_id': 'req-2c918273-0943-439e-8418-07f701ce7b47'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:22.173385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd329d951b42740b88a3782cd08961ffb', 'request_id': 'req-2c918273-0943-439e-8418-07f701ce7b47'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:22.186559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 685/1381] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:22.189440 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ecbc5-dc8d-4fce-bc62-9bb6dbe44e43 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:22.203949 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd329d951b42740b88a3782cd08961ffb', 'request_id': 'req-2c918273-0943-439e-8418-07f701ce7b47'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:22.227766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ecbc5-dc8d-4fce-bc62-9bb6dbe44e43 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:22.351812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] The token cache is being invalidate because project d329d951b42740b88a3782cd08961ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:22.352113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:22.353560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c918273-0943-439e-8418-07f701ce7b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:22.356699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 697/1382] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:21 2022] DELETE /identity/v3/projects/d329d951b42740b88a3782cd08961ffb => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:22.363257 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece6c74-23e6-4e39-b6d2-4814f279d9f0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:22.363546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece6c74-23e6-4e39-b6d2-4814f279d9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:22.363820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece6c74-23e6-4e39-b6d2-4814f279d9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:22.418179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:22.419550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:22.420919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:22.421377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:22.422290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:22.426590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:22.431346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-eece6c74-23e6-4e39-b6d2-4814f279d9f0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:22.432334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-eece6c74-23e6-4e39-b6d2-4814f279d9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:22.442975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 686/1383] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:22.443848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:22.448911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 698/1384] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:22.449927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:22.457669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db451c-56f9-41d4-9f96-6dc5c245b368 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:22.458031 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db451c-56f9-41d4-9f96-6dc5c245b368 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:22.458249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db451c-56f9-41d4-9f96-6dc5c245b368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:22.545907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92f809-c605-4571-9671-7736bda06ba8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:22.551783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:22.552981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:22.553274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:22.553560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:22.556202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:22.584362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a0db451c-56f9-41d4-9f96-6dc5c245b368 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:22.585543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a0db451c-56f9-41d4-9f96-6dc5c245b368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:22.588176 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:22.633609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:22.635516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:22.655832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 687/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:22.655832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:22.655832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:22.672909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73823a58-d1e1-462f-bdb3-c0110adbe443 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:22.679627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b275497-5225-45e5-af06-a58b30914401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:22.680978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b275497-5225-45e5-af06-a58b30914401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:22.681287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b275497-5225-45e5-af06-a58b30914401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:22.681616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b275497-5225-45e5-af06-a58b30914401 admin admin] PATH_INFO: `/v3/projects/4856d9f3058c49b2ae56c3e95d878684/users/09622c9333e749f99a5f9fdfefc3f91f/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:22.693822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 699/1386] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:22.695072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2689d05-7dca-4133-b3f0-dee6505da47e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:22.695500 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2689d05-7dca-4133-b3f0-dee6505da47e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:22.695778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2689d05-7dca-4133-b3f0-dee6505da47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:22.705902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b275497-5225-45e5-af06-a58b30914401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4856d9f3058c49b2ae56c3e95d878684, user_id=09622c9333e749f99a5f9fdfefc3f91f, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:22.708671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b275497-5225-45e5-af06-a58b30914401 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:22.741809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e2689d05-7dca-4133-b3f0-dee6505da47e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:22.743084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e2689d05-7dca-4133-b3f0-dee6505da47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:22.762310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 700/1387] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:22.762310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2689d05-7dca-4133-b3f0-dee6505da47e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:22.762310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e2689d05-7dca-4133-b3f0-dee6505da47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:22.774536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2689d05-7dca-4133-b3f0-dee6505da47e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:22.780150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:22.781466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:22.781466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:22.781645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] PATH_INFO: `/v3/projects/f427efac014644dea26de8e1f49121a4` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:22.782858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f427efac014644dea26de8e1f49121a4)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:22.786914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:22.825862 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 688/1388] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:22 2022] PUT /identity/v3/projects/4856d9f3058c49b2ae56c3e95d878684/users/09622c9333e749f99a5f9fdfefc3f91f/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:22.827179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b275497-5225-45e5-af06-a58b30914401 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:22.954028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b275497-5225-45e5-af06-a58b30914401 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:22.954612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f427efac014644dea26de8e1f49121a4', 'request_id': 'req-c1291e84-9853-4000-a12f-60ce4d2f5f56'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:22.961838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f427efac014644dea26de8e1f49121a4', 'request_id': 'req-c1291e84-9853-4000-a12f-60ce4d2f5f56'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:22.972433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f427efac014644dea26de8e1f49121a4', 'request_id': 'req-c1291e84-9853-4000-a12f-60ce4d2f5f56'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:23.093128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29db1f-756c-40e6-b0c2-99e29e4afdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:23.094039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29db1f-756c-40e6-b0c2-99e29e4afdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:23.094532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29db1f-756c-40e6-b0c2-99e29e4afdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:23.094950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29db1f-756c-40e6-b0c2-99e29e4afdb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:23.095780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29db1f-756c-40e6-b0c2-99e29e4afdb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:23.096243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29db1f-756c-40e6-b0c2-99e29e4afdb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:23.098316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29db1f-756c-40e6-b0c2-99e29e4afdb5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:23.113827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 689/1389] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:38:23.115769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99873e5-3cb4-4274-abed-2d6f88b6ad56 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:23.116061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99873e5-3cb4-4274-abed-2d6f88b6ad56 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:23.116334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99873e5-3cb4-4274-abed-2d6f88b6ad56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:23.130174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] The token cache is being invalidate because project f427efac014644dea26de8e1f49121a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:23.130914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:23.131765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:23.134081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 701/1390] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:22 2022] DELETE /identity/v3/projects/f427efac014644dea26de8e1f49121a4 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:23.135092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:23.137436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:23.153905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e99873e5-3cb4-4274-abed-2d6f88b6ad56 None None] MFA Rules not processed for user `09622c9333e749f99a5f9fdfefc3f91f`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:23.155387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e99873e5-3cb4-4274-abed-2d6f88b6ad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:23.272430 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 690/1391] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:23.272430 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99873e5-3cb4-4274-abed-2d6f88b6ad56 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:23.272660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e99873e5-3cb4-4274-abed-2d6f88b6ad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:23.297788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99873e5-3cb4-4274-abed-2d6f88b6ad56 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:23.313569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:23.314339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:23.315216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:23.315601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:23.316358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:23.329214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:23.378659 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1291e84-9853-4000-a12f-60ce4d2f5f56 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:23.399659 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:23.400526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:23.400988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:23.401384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:23.406637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:23.474622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 691/1392] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:23 2022] POST /identity/v3/projects => generated 377 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:23.475478 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:23.479082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:23.505416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:23.508334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:23.522889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 702/1393] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:38:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:38:23.525589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:23.529885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:23.560150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7f209-7d1e-4fe2-8b81-db18102bbbb8 service swift] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:23.605621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dbaa8-451e-45b2-9f46-8d445741234b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:23.621504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:23.622441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:23.622796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:23.623177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] PATH_INFO: `/v3/users/cc0a0200fd5a4de18f0d51283b45a750` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:23.629323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0a0200fd5a4de18f0d51283b45a750)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:23.632815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:23.746313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d6a5b-ad74-41a0-844d-837d40affd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:23.751377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d6a5b-ad74-41a0-844d-837d40affd47 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:23.752991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d6a5b-ad74-41a0-844d-837d40affd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:23.753524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d6a5b-ad74-41a0-844d-837d40affd47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:23.756625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d6a5b-ad74-41a0-844d-837d40affd47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:23.764046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d6a5b-ad74-41a0-844d-837d40affd47 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:23.848515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:23.913058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 703/1394] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:23 2022] POST /identity/v3/projects => generated 355 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:23.914401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d6a5b-ad74-41a0-844d-837d40affd47 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:23.971750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0a0200fd5a4de18f0d51283b45a750'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:24.023073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0a0200fd5a4de18f0d51283b45a750'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:24.082439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d6a5b-ad74-41a0-844d-837d40affd47 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:24.088706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:24.089738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:24.090322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:24.090699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:24.091751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:24.097995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:24.192960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0a0200fd5a4de18f0d51283b45a750'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:24.193723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0a0200fd5a4de18f0d51283b45a750'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:24.206070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0a0200fd5a4de18f0d51283b45a750'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:24.209308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 704/1395] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:23 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:24.210680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:24.218929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:24.223866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 692/1396] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:23 2022] DELETE /identity/v3/users/cc0a0200fd5a4de18f0d51283b45a750 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:24.226301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:24.229725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:24.248429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2527a25-3456-47e9-89d4-533c85404ad4 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:24.505282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032ba9d-63ae-4b5b-83ca-a33a4ae597f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:24.506074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032ba9d-63ae-4b5b-83ca-a33a4ae597f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:24.507392 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032ba9d-63ae-4b5b-83ca-a33a4ae597f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:24.507675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032ba9d-63ae-4b5b-83ca-a33a4ae597f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:24.508607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032ba9d-63ae-4b5b-83ca-a33a4ae597f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:24.515066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0396918-3517-4ba4-88d7-f1047ecdc50c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:24.525877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032ba9d-63ae-4b5b-83ca-a33a4ae597f7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:24.538957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122be02c-fa84-4961-8fd5-715e8617b7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:24.546498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122be02c-fa84-4961-8fd5-715e8617b7de admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:24.546924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122be02c-fa84-4961-8fd5-715e8617b7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:24.547207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122be02c-fa84-4961-8fd5-715e8617b7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:24.550434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122be02c-fa84-4961-8fd5-715e8617b7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:24.556878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122be02c-fa84-4961-8fd5-715e8617b7de admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:24.595306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 693/1397] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:24.599502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122be02c-fa84-4961-8fd5-715e8617b7de admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:24.655989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122be02c-fa84-4961-8fd5-715e8617b7de admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:24.688321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa7b8-6d62-4232-a3b0-a6ddf47f01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:24.690446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa7b8-6d62-4232-a3b0-a6ddf47f01d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:24.691495 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa7b8-6d62-4232-a3b0-a6ddf47f01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:24.691800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaa7b8-6d62-4232-a3b0-a6ddf47f01d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:24.693492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaa7b8-6d62-4232-a3b0-a6ddf47f01d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:24.698164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaa7b8-6d62-4232-a3b0-a6ddf47f01d6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:24.735534 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 705/1398] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:24 2022] POST /identity/v3/users => generated 284 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:24.740498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032ba9d-63ae-4b5b-83ca-a33a4ae597f7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:24.758441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 694/1399] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:24.760866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa7b8-6d62-4232-a3b0-a6ddf47f01d6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:24.825245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032ba9d-63ae-4b5b-83ca-a33a4ae597f7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:24.831702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaa7b8-6d62-4232-a3b0-a6ddf47f01d6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:24.838647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cf99b-cd7a-4432-b0c3-2305489d1988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:24.839542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cf99b-cd7a-4432-b0c3-2305489d1988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:24.839830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cf99b-cd7a-4432-b0c3-2305489d1988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:24.840106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cf99b-cd7a-4432-b0c3-2305489d1988 admin admin] PATH_INFO: `/v3/projects/0ef7d8196e32442585ae469abb685fd6/users/f5a6310201614d09939c820f49a5f2cb/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:24.860271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e7e9b-67dd-4c4e-84b3-3a303f161d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:24.867860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e7e9b-67dd-4c4e-84b3-3a303f161d79 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:24.867860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e7e9b-67dd-4c4e-84b3-3a303f161d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:24.867860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e7e9b-67dd-4c4e-84b3-3a303f161d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:24.872256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e7e9b-67dd-4c4e-84b3-3a303f161d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:24.891012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e7e9b-67dd-4c4e-84b3-3a303f161d79 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:24.905703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cf99b-cd7a-4432-b0c3-2305489d1988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef7d8196e32442585ae469abb685fd6, user_id=f5a6310201614d09939c820f49a5f2cb, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:24.908904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cf99b-cd7a-4432-b0c3-2305489d1988 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:24.947144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 706/1400] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:24.950003 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbab41-7a66-4118-b5e0-bf67a5dda95f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:24.950491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbab41-7a66-4118-b5e0-bf67a5dda95f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:24.950998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbab41-7a66-4118-b5e0-bf67a5dda95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:25.017199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 695/1401] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:24 2022] PUT /identity/v3/projects/0ef7d8196e32442585ae469abb685fd6/users/f5a6310201614d09939c820f49a5f2cb/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:25.019384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cf99b-cd7a-4432-b0c3-2305489d1988 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:25.090291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d3dbab41-7a66-4118-b5e0-bf67a5dda95f None None] MFA Rules not processed for user `db04575738b04f2d95803498e19554c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:25.093014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d3dbab41-7a66-4118-b5e0-bf67a5dda95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:25.265468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cf99b-cd7a-4432-b0c3-2305489d1988 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:25.284822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60adaaf-e346-4550-9d54-b5c38358600a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:25.286549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60adaaf-e346-4550-9d54-b5c38358600a admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:25.287005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60adaaf-e346-4550-9d54-b5c38358600a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:25.287442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60adaaf-e346-4550-9d54-b5c38358600a admin admin] PATH_INFO: `/v3/projects/eaefb32081c64da88c529c889a958419/users/5ae17868764249fd8eb656dc8ddcb08a/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:25.354887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60adaaf-e346-4550-9d54-b5c38358600a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaefb32081c64da88c529c889a958419, user_id=5ae17868764249fd8eb656dc8ddcb08a, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:25.358117 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60adaaf-e346-4550-9d54-b5c38358600a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:25.404005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 707/1402] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Oct 12 12:38:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:25.409022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04810234-cfd4-4a54-935b-1bf031d25a85 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:25.422575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04810234-cfd4-4a54-935b-1bf031d25a85 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:25.423139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04810234-cfd4-4a54-935b-1bf031d25a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:25.500182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-04810234-cfd4-4a54-935b-1bf031d25a85 None None] MFA Rules not processed for user `f5a6310201614d09939c820f49a5f2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:25.501568 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-04810234-cfd4-4a54-935b-1bf031d25a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:25.579257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:25.671631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 696/1403] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:25 2022] PUT /identity/v3/projects/eaefb32081c64da88c529c889a958419/users/5ae17868764249fd8eb656dc8ddcb08a/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:25.672516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 708/1404] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:25.673552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60adaaf-e346-4550-9d54-b5c38358600a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:25.675184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04810234-cfd4-4a54-935b-1bf031d25a85 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:25.708858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04810234-cfd4-4a54-935b-1bf031d25a85 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:25.890384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:25.892140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:25.892530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:25.893116 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:25.894373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:25.897954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:25.932857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 709/1405] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:25.937647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:25.943813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:25.948404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60adaaf-e346-4550-9d54-b5c38358600a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:25.974037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c16760-4b51-47b4-aab0-40f2ed055b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:25.974037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c16760-4b51-47b4-aab0-40f2ed055b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:25.974037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c16760-4b51-47b4-aab0-40f2ed055b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:25.974037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c16760-4b51-47b4-aab0-40f2ed055b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:25.974423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c16760-4b51-47b4-aab0-40f2ed055b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:25.974423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c16760-4b51-47b4-aab0-40f2ed055b4a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:26.121500 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a040d34-169f-48da-8972-e40234b24948 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:26.135167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809f4d0-e6dc-4a5a-9226-ff9094f39151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:26.135167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809f4d0-e6dc-4a5a-9226-ff9094f39151 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:26.135167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809f4d0-e6dc-4a5a-9226-ff9094f39151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:26.135167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809f4d0-e6dc-4a5a-9226-ff9094f39151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:26.144712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809f4d0-e6dc-4a5a-9226-ff9094f39151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:26.146260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809f4d0-e6dc-4a5a-9226-ff9094f39151 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:26.166015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 710/1406] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:26.167495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809f4d0-e6dc-4a5a-9226-ff9094f39151 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:26.211055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809f4d0-e6dc-4a5a-9226-ff9094f39151 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:26.215370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 697/1407] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:25 2022] POST /identity/v3/projects => generated 381 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:26.220562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c16760-4b51-47b4-aab0-40f2ed055b4a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:26.485624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c16760-4b51-47b4-aab0-40f2ed055b4a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:26.520761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e911c-cb83-40d5-8ec7-37942f387f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:26.520761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e911c-cb83-40d5-8ec7-37942f387f5d admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:26.520761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e911c-cb83-40d5-8ec7-37942f387f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:26.528036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83a1c4-9b44-4c04-9496-24c63f88c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:26.529492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83a1c4-9b44-4c04-9496-24c63f88c6f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:26.532426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83a1c4-9b44-4c04-9496-24c63f88c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:26.533158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83a1c4-9b44-4c04-9496-24c63f88c6f1 admin admin] PATH_INFO: `/v3/projects/eaefb32081c64da88c529c889a958419/users/5ae17868764249fd8eb656dc8ddcb08a/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:26.540332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e911c-cb83-40d5-8ec7-37942f387f5d admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:26.543791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83a1c4-9b44-4c04-9496-24c63f88c6f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaefb32081c64da88c529c889a958419, user_id=5ae17868764249fd8eb656dc8ddcb08a, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:26.544448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e911c-cb83-40d5-8ec7-37942f387f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:26.554128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e911c-cb83-40d5-8ec7-37942f387f5d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:26.562149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83a1c4-9b44-4c04-9496-24c63f88c6f1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:26.639529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 711/1408] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:26 2022] PUT /identity/v3/projects/eaefb32081c64da88c529c889a958419/users/5ae17868764249fd8eb656dc8ddcb08a/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:26.651911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2cc7b-5be9-44e3-b61c-f460fa5f2a96 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:26.652158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2cc7b-5be9-44e3-b61c-f460fa5f2a96 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:26.652433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2cc7b-5be9-44e3-b61c-f460fa5f2a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:26.720638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 698/1409] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:26 2022] POST /identity/v3/users => generated 297 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:26.725400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e911c-cb83-40d5-8ec7-37942f387f5d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:26.735034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:26.764323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e911c-cb83-40d5-8ec7-37942f387f5d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:26.779220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-96a2cc7b-5be9-44e3-b61c-f460fa5f2a96 None None] MFA Rules not processed for user `5ae17868764249fd8eb656dc8ddcb08a`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:26.787418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-96a2cc7b-5be9-44e3-b61c-f460fa5f2a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:26.987622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:26.988275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:26.988555 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:26.988824 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:26.989593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:26.992727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:27.020848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 699/1410] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:27.022074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:27.032278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:27.053772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e8342-712e-4ca2-be89-33d407239155 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:27.067431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 712/1411] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:27.067647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:27.067647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:27.067809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2cc7b-5be9-44e3-b61c-f460fa5f2a96 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:27.067883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:27.067883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] PATH_INFO: `/v3/projects/b35637e82dbb44feaeb318f61075a386` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:27.068227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35637e82dbb44feaeb318f61075a386)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:27.075395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:27.105781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2cc7b-5be9-44e3-b61c-f460fa5f2a96 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:27.105781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:27.122293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:27.122664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:27.122944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] PATH_INFO: `/v3/projects/af344f72f4f443ff92df847143b494d5/users/35f030aca17b4dc58548a25d5b3b89c7/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:27.203220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af344f72f4f443ff92df847143b494d5, user_id=35f030aca17b4dc58548a25d5b3b89c7, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:27.212358 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:27.388686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 713/1412] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:27 2022] PUT /identity/v3/projects/af344f72f4f443ff92df847143b494d5/users/35f030aca17b4dc58548a25d5b3b89c7/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:27.389832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:27.408450 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:27.428464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35637e82dbb44feaeb318f61075a386', 'request_id': 'req-ea1fbe17-3387-459c-a918-c53858dc30e0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:27.449058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35637e82dbb44feaeb318f61075a386', 'request_id': 'req-ea1fbe17-3387-459c-a918-c53858dc30e0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:27.468196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35637e82dbb44feaeb318f61075a386', 'request_id': 'req-ea1fbe17-3387-459c-a918-c53858dc30e0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:27.597094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4453d-c7e1-4865-a922-34df5a5bcaf9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:27.689074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] The token cache is being invalidate because project b35637e82dbb44feaeb318f61075a386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:27.694746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:27.697617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:27.702041 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 700/1413] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:27 2022] DELETE /identity/v3/projects/b35637e82dbb44feaeb318f61075a386 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:27.705563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:27.712329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:27.826980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:27.826980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:27.826980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:27.826980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:27.829084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:27.944660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1fbe17-3387-459c-a918-c53858dc30e0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:27.956374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a502f-d261-4791-9fe0-d8d3c5ca320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:27.957209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a502f-d261-4791-9fe0-d8d3c5ca320f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:27.957608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a502f-d261-4791-9fe0-d8d3c5ca320f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:27.958174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a502f-d261-4791-9fe0-d8d3c5ca320f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:27.960049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a502f-d261-4791-9fe0-d8d3c5ca320f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:27.961811 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a502f-d261-4791-9fe0-d8d3c5ca320f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:27.994871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 701/1414] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:27.997860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2f5fc-19f9-43a8-9ba3-00118fa93132 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:27.999521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2f5fc-19f9-43a8-9ba3-00118fa93132 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:28.002948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2f5fc-19f9-43a8-9ba3-00118fa93132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:28.025082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:28.027556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:28.046661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4ea2f5fc-19f9-43a8-9ba3-00118fa93132 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:28.051508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4ea2f5fc-19f9-43a8-9ba3-00118fa93132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:28.056270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 714/1415] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:28.056270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:28.070369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b49e-758d-49b7-a506-4302e0f72b91 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:28.079238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802755e-acc5-4b5c-8be3-37c1bf66758a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:28.080083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802755e-acc5-4b5c-8be3-37c1bf66758a admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:28.081285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802755e-acc5-4b5c-8be3-37c1bf66758a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:28.083359 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802755e-acc5-4b5c-8be3-37c1bf66758a admin admin] PATH_INFO: `/v3/projects/af344f72f4f443ff92df847143b494d5/users/35f030aca17b4dc58548a25d5b3b89c7/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:28.086137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 702/1416] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:28.086577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b802755e-acc5-4b5c-8be3-37c1bf66758a admin admin] RBAC: Authorizing `identity:create_grant(project_id=af344f72f4f443ff92df847143b494d5, user_id=35f030aca17b4dc58548a25d5b3b89c7, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:28.090406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b802755e-acc5-4b5c-8be3-37c1bf66758a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:28.094202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c1a5-f047-4965-a29a-b10eab1dcc64 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:28.094510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c1a5-f047-4965-a29a-b10eab1dcc64 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:28.095004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c1a5-f047-4965-a29a-b10eab1dcc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:28.100401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:28.110409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 715/1417] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:28 2022] PUT /identity/v3/projects/af344f72f4f443ff92df847143b494d5/users/35f030aca17b4dc58548a25d5b3b89c7/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:28.122039 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1730d-2654-4933-877d-0c5a78d3ad57 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:28.122785 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1730d-2654-4933-877d-0c5a78d3ad57 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:28.123211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1730d-2654-4933-877d-0c5a78d3ad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:28.179807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f081c1a5-f047-4965-a29a-b10eab1dcc64 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:28.182897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f081c1a5-f047-4965-a29a-b10eab1dcc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:28.254877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-3ad1730d-2654-4933-877d-0c5a78d3ad57 None None] MFA Rules not processed for user `35f030aca17b4dc58548a25d5b3b89c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:28.270713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3ad1730d-2654-4933-877d-0c5a78d3ad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:28.492916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 703/1418] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:28.495816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081c1a5-f047-4965-a29a-b10eab1dcc64 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:28.499234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f081c1a5-f047-4965-a29a-b10eab1dcc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:28.530508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 716/1419] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:28.531718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1730d-2654-4933-877d-0c5a78d3ad57 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:28.559120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1730d-2654-4933-877d-0c5a78d3ad57 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:28.580082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:28.581904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:28.583510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:28.585222 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:28.586186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:28.587459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:28.593758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:28.633577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 717/1420] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:38:28.633577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:28.672867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ecd1e-3929-4b99-9435-7f115224264d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:28.688634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:28.690376 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:28.692237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:28.692609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:28.695770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:28.699901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:28.732084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081c1a5-f047-4965-a29a-b10eab1dcc64 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:28.743179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:28.744856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:28.745336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:28.746398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:28.749974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:28.783817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:28.827490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 718/1421] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:28 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:28.828521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:28.834448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:28.991449 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:28.992918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:29.006643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5807d63-1f18-410b-bfe0-c853d60de6d7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:29.011982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 704/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:29.013303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:29.015927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d7707-e0c0-48c4-bd6c-f3e9994838ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:29.016587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d7707-e0c0-48c4-bd6c-f3e9994838ae service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:29.016865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d7707-e0c0-48c4-bd6c-f3e9994838ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:29.018351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d7707-e0c0-48c4-bd6c-f3e9994838ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:29.037884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ed935-30bf-4f72-bd60-f07fd82e1ff7 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:29.179143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:29.248063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d7707-e0c0-48c4-bd6c-f3e9994838ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:29.250298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d7707-e0c0-48c4-bd6c-f3e9994838ae service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:29.261242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eebaf-c13c-461a-b4a8-1d59521156dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:29.262198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eebaf-c13c-461a-b4a8-1d59521156dc admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:29.262479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eebaf-c13c-461a-b4a8-1d59521156dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:29.262768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eebaf-c13c-461a-b4a8-1d59521156dc admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:29.265737 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164eebaf-c13c-461a-b4a8-1d59521156dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:29.270340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164eebaf-c13c-461a-b4a8-1d59521156dc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:29.279241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 719/1423] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:29.370933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 705/1424] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:29 2022] POST /identity/v3/users => generated 293 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:29.376810 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d7707-e0c0-48c4-bd6c-f3e9994838ae service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:29.396514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d7707-e0c0-48c4-bd6c-f3e9994838ae service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:29.404944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccd8b0-6706-4337-9289-e099f1dece8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:29.406569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccd8b0-6706-4337-9289-e099f1dece8a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:29.406888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccd8b0-6706-4337-9289-e099f1dece8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:29.407411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccd8b0-6706-4337-9289-e099f1dece8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:29.409279 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccd8b0-6706-4337-9289-e099f1dece8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:29.413476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccd8b0-6706-4337-9289-e099f1dece8a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:29.452029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 720/1425] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:29.457954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eebaf-c13c-461a-b4a8-1d59521156dc admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:29.469908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eebaf-c13c-461a-b4a8-1d59521156dc admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:29.484532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93566fc3-413d-481d-b7e8-0509f7d22038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:29.503364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566fc3-413d-481d-b7e8-0509f7d22038 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:29.503364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566fc3-413d-481d-b7e8-0509f7d22038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:29.503364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93566fc3-413d-481d-b7e8-0509f7d22038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:29.503364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93566fc3-413d-481d-b7e8-0509f7d22038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:29.503364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93566fc3-413d-481d-b7e8-0509f7d22038 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:29.548144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccd8b0-6706-4337-9289-e099f1dece8a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:29.555217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 706/1426] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:29.566210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccd8b0-6706-4337-9289-e099f1dece8a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:29.574185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035b129-66d5-40ad-8101-3a81fe491ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:29.574875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b129-66d5-40ad-8101-3a81fe491ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:29.575154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b129-66d5-40ad-8101-3a81fe491ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:29.575432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b129-66d5-40ad-8101-3a81fe491ee5 admin admin] PATH_INFO: `/v3/projects/82be8c6efc0e4fe388d5030d5884e38f/users/7ce6ed02f0bb485995f42b5aaa6a61f8/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:29.657123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035b129-66d5-40ad-8101-3a81fe491ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82be8c6efc0e4fe388d5030d5884e38f, user_id=7ce6ed02f0bb485995f42b5aaa6a61f8, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:29.660169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035b129-66d5-40ad-8101-3a81fe491ee5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:29.705734 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:29.735348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93566fc3-413d-481d-b7e8-0509f7d22038 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:29.764218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 721/1427] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:29 2022] PUT /identity/v3/projects/82be8c6efc0e4fe388d5030d5884e38f/users/7ce6ed02f0bb485995f42b5aaa6a61f8/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:29.770531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748140-9cc0-49d6-b9fe-6faff540f074 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:29.775228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93566fc3-413d-481d-b7e8-0509f7d22038 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:29.776834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748140-9cc0-49d6-b9fe-6faff540f074 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:29.779948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748140-9cc0-49d6-b9fe-6faff540f074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:29.893613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-cd748140-9cc0-49d6-b9fe-6faff540f074 None None] MFA Rules not processed for user `7ce6ed02f0bb485995f42b5aaa6a61f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:29.914324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-cd748140-9cc0-49d6-b9fe-6faff540f074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:29.956579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8150a-0778-4c4c-9f89-3e233fd36151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:29.958154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8150a-0778-4c4c-9f89-3e233fd36151 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:29.958446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8150a-0778-4c4c-9f89-3e233fd36151 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:29.960236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8150a-0778-4c4c-9f89-3e233fd36151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:29.966416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-05e8150a-0778-4c4c-9f89-3e233fd36151 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:30.160840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 722/1428] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:30.163246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd748140-9cc0-49d6-b9fe-6faff540f074 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:30.247855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8150a-0778-4c4c-9f89-3e233fd36151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:30.248593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8150a-0778-4c4c-9f89-3e233fd36151 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:30.292015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 707/1429] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:30.297332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24a11-d5ff-49a6-9a02-439249e0c9cc None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:30.298535 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24a11-d5ff-49a6-9a02-439249e0c9cc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:30.299504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24a11-d5ff-49a6-9a02-439249e0c9cc None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:30.301374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 708/1430] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:38:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:38:30.315816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd748140-9cc0-49d6-b9fe-6faff540f074 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:30.341215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:30.346722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30c718-7c1b-49e2-a0f7-c8a258b1403d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:30.347370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30c718-7c1b-49e2-a0f7-c8a258b1403d admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:30.347653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30c718-7c1b-49e2-a0f7-c8a258b1403d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:30.347929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30c718-7c1b-49e2-a0f7-c8a258b1403d admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:30.348582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30c718-7c1b-49e2-a0f7-c8a258b1403d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:30.350609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30c718-7c1b-49e2-a0f7-c8a258b1403d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:30.375811 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc24a11-d5ff-49a6-9a02-439249e0c9cc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:30.379847 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ffc24a11-d5ff-49a6-9a02-439249e0c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:30.441529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc24a11-d5ff-49a6-9a02-439249e0c9cc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:30.474395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:30.475201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:30.479995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:30.480402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] PATH_INFO: `/v3/users/156ae60d9b5e47f9aab9b02c180fb642` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:30.481925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=156ae60d9b5e47f9aab9b02c180fb642)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:30.484153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:30.546602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 723/1431] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:30 2022] POST /identity/v3/projects => generated 381 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:30.551437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f76d1-dce4-480d-adba-50f1f369e20a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:30.556186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f76d1-dce4-480d-adba-50f1f369e20a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:30.556673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f76d1-dce4-480d-adba-50f1f369e20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:30.629044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-021f76d1-dce4-480d-adba-50f1f369e20a None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:30.630470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-021f76d1-dce4-480d-adba-50f1f369e20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:30.683256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:30.751360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156ae60d9b5e47f9aab9b02c180fb642'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:30.813040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156ae60d9b5e47f9aab9b02c180fb642'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:30.847336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 724/1432] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:38:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:30.848430 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f76d1-dce4-480d-adba-50f1f369e20a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:30.876684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156ae60d9b5e47f9aab9b02c180fb642'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:30.877775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156ae60d9b5e47f9aab9b02c180fb642'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:30.887590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d61d7bbc-9f49-4ff9-8999-37fd101b2484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156ae60d9b5e47f9aab9b02c180fb642'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:30.906607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 709/1433] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:30 2022] DELETE /identity/v3/users/156ae60d9b5e47f9aab9b02c180fb642 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:30.909096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e966-7eee-4d90-8e61-c2ed41b58962 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:30.909645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e966-7eee-4d90-8e61-c2ed41b58962 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:30.910152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3e966-7eee-4d90-8e61-c2ed41b58962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:30.957566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f1d3e966-7eee-4d90-8e61-c2ed41b58962 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:30.961889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f1d3e966-7eee-4d90-8e61-c2ed41b58962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:31.044193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f76d1-dce4-480d-adba-50f1f369e20a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:31.137496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 710/1434] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:31.147121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3e966-7eee-4d90-8e61-c2ed41b58962 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:31.151897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f1d3e966-7eee-4d90-8e61-c2ed41b58962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:31.204877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:31.206398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:31.206779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:31.209431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:31.210425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:31.216749 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:31.290760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3e966-7eee-4d90-8e61-c2ed41b58962 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:31.300460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:31.301460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:31.302140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:31.302645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:31.305362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:31.331577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:31.333565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:31.351540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 725/1435] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:30 2022] POST /identity/v3/users => generated 297 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:31.352528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:31.356479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:31.362567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 711/1436] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:31.363824 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:31.365577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:31.387050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14768c-05f5-4a0f-87c2-866147bb2a28 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:31.390931 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60214e0-7a02-4fba-81e8-fe8b86b87171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:31.391883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60214e0-7a02-4fba-81e8-fe8b86b87171 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:31.392305 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60214e0-7a02-4fba-81e8-fe8b86b87171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:31.393408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60214e0-7a02-4fba-81e8-fe8b86b87171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:31.394327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60214e0-7a02-4fba-81e8-fe8b86b87171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:31.396263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60214e0-7a02-4fba-81e8-fe8b86b87171 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:31.414053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 712/1437] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:31.415002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60214e0-7a02-4fba-81e8-fe8b86b87171 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:31.426593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60214e0-7a02-4fba-81e8-fe8b86b87171 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:31.433979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addba573-d8d2-41cc-acb6-ef8d055da75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:31.434935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addba573-d8d2-41cc-acb6-ef8d055da75f admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:31.444277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addba573-d8d2-41cc-acb6-ef8d055da75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:31.444277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addba573-d8d2-41cc-acb6-ef8d055da75f admin admin] PATH_INFO: `/v3/projects/bf34f926952f4b64bb888efcecadb394/users/ce37c84b3e88462082a9080b2762edf3/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:31.456860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addba573-d8d2-41cc-acb6-ef8d055da75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf34f926952f4b64bb888efcecadb394, user_id=ce37c84b3e88462082a9080b2762edf3, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:31.460173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addba573-d8d2-41cc-acb6-ef8d055da75f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:31.483266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472ec92-aab3-4200-8ec4-4bbc61d14919 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:31.493505 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01c903-f108-46e0-ae57-d2e44ab0d473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:31.494746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01c903-f108-46e0-ae57-d2e44ab0d473 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:31.496171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01c903-f108-46e0-ae57-d2e44ab0d473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:31.496575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01c903-f108-46e0-ae57-d2e44ab0d473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:31.521985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01c903-f108-46e0-ae57-d2e44ab0d473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:31.526098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01c903-f108-46e0-ae57-d2e44ab0d473 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:31.555991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 726/1438] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:31.607312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:31.611563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 713/1439] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:31 2022] PUT /identity/v3/projects/bf34f926952f4b64bb888efcecadb394/users/ce37c84b3e88462082a9080b2762edf3/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:31.615689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01c903-f108-46e0-ae57-d2e44ab0d473 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:31.632737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01c903-f108-46e0-ae57-d2e44ab0d473 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:31.764837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477046a-7cab-41cb-9e98-c64401fff4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:31.765489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477046a-7cab-41cb-9e98-c64401fff4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:31.765769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477046a-7cab-41cb-9e98-c64401fff4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:31.766065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477046a-7cab-41cb-9e98-c64401fff4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:31.770323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477046a-7cab-41cb-9e98-c64401fff4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:31.776134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477046a-7cab-41cb-9e98-c64401fff4b6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:31.852076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addba573-d8d2-41cc-acb6-ef8d055da75f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:31.852455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 727/1440] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:31.869583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addba573-d8d2-41cc-acb6-ef8d055da75f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:31.878141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:31.879222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:31.879617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:31.880097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] PATH_INFO: `/v3/projects/bf34f926952f4b64bb888efcecadb394/users/ce37c84b3e88462082a9080b2762edf3/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:31.883602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf34f926952f4b64bb888efcecadb394, user_id=ce37c84b3e88462082a9080b2762edf3, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:31.887021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:31.895798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:31.927829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 714/1441] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:31 2022] PUT /identity/v3/projects/bf34f926952f4b64bb888efcecadb394/users/ce37c84b3e88462082a9080b2762edf3/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:31.930095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d70d5-b63e-4674-8d19-f442cae197c0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:31.930510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d70d5-b63e-4674-8d19-f442cae197c0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:31.930896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d70d5-b63e-4674-8d19-f442cae197c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:31.977604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-312d70d5-b63e-4674-8d19-f442cae197c0 None None] MFA Rules not processed for user `ce37c84b3e88462082a9080b2762edf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:31.979784 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-312d70d5-b63e-4674-8d19-f442cae197c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:32.175384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 728/1442] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:32.188303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:32.197816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:32.372627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493baba-85c1-42b6-8b55-58aec6989083 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:32.385581 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84784a-faaf-4175-a238-48d1a53491e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:32.388202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84784a-faaf-4175-a238-48d1a53491e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:32.390243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84784a-faaf-4175-a238-48d1a53491e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:32.390881 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84784a-faaf-4175-a238-48d1a53491e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:32.424016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84784a-faaf-4175-a238-48d1a53491e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:32.427546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84784a-faaf-4175-a238-48d1a53491e4 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:32.462494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 715/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:32.606663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d70d5-b63e-4674-8d19-f442cae197c0 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:32.781211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d70d5-b63e-4674-8d19-f442cae197c0 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:32.789010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:32.789975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:32.790900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:32.791440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] PATH_INFO: `/v3/users/7ce6ed02f0bb485995f42b5aaa6a61f8` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:32.793771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce6ed02f0bb485995f42b5aaa6a61f8)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:32.795868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:33.109634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce6ed02f0bb485995f42b5aaa6a61f8'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.136045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce6ed02f0bb485995f42b5aaa6a61f8'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.137288 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce6ed02f0bb485995f42b5aaa6a61f8'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.248713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce6ed02f0bb485995f42b5aaa6a61f8'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.261686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce6ed02f0bb485995f42b5aaa6a61f8'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.272451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 729/1444] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:32 2022] DELETE /identity/v3/users/7ce6ed02f0bb485995f42b5aaa6a61f8 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:33.277983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d489ed-4102-415b-838d-e5f429e4a279 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:33.278518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d489ed-4102-415b-838d-e5f429e4a279 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:33.279004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d489ed-4102-415b-838d-e5f429e4a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:33.304236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-63d489ed-4102-415b-838d-e5f429e4a279 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:33.305906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-63d489ed-4102-415b-838d-e5f429e4a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:33.390663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 716/1445] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:33.481962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:33.483506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:33.577401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d489ed-4102-415b-838d-e5f429e4a279 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:33.578947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-63d489ed-4102-415b-838d-e5f429e4a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:33.595664 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d489ed-4102-415b-838d-e5f429e4a279 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:33.602915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:33.603823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:33.604114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:33.608105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] PATH_INFO: `/v3/projects/eb6288104d594741b1270fac948beae9` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:33.609959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb6288104d594741b1270fac948beae9)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:33.613015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:33.638453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b908aa-7535-461c-be71-9a715538d728 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:33.643535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c637aa-0b23-4f8d-b0ba-15ad618ff854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:33.644409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c637aa-0b23-4f8d-b0ba-15ad618ff854 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:33.647543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c637aa-0b23-4f8d-b0ba-15ad618ff854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:33.647543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c637aa-0b23-4f8d-b0ba-15ad618ff854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:33.669959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c637aa-0b23-4f8d-b0ba-15ad618ff854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:33.671510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c637aa-0b23-4f8d-b0ba-15ad618ff854 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:33.701058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 730/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:33.786036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb6288104d594741b1270fac948beae9', 'request_id': 'req-0b72570e-50d2-488f-a895-bd38b5fab4a6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.799321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb6288104d594741b1270fac948beae9', 'request_id': 'req-0b72570e-50d2-488f-a895-bd38b5fab4a6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.804514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb6288104d594741b1270fac948beae9', 'request_id': 'req-0b72570e-50d2-488f-a895-bd38b5fab4a6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.934821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] The token cache is being invalidate because project eb6288104d594741b1270fac948beae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:33.934821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.934821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0b72570e-50d2-488f-a895-bd38b5fab4a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:33.936206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 717/1447] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:33 2022] DELETE /identity/v3/projects/eb6288104d594741b1270fac948beae9 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:33.939429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c637aa-0b23-4f8d-b0ba-15ad618ff854 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:33.970903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c637aa-0b23-4f8d-b0ba-15ad618ff854 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:33.996108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57d06c-3cf6-47e7-9fe9-60342c0014f6 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:33.996399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57d06c-3cf6-47e7-9fe9-60342c0014f6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:33.996677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57d06c-3cf6-47e7-9fe9-60342c0014f6 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:33.998072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 718/1448] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:38:34.130776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:34.131751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:34.133150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:34.134352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] PATH_INFO: `/v3/users/7598284269a0410ba7872901531f8203` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:34.138466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7598284269a0410ba7872901531f8203)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:34.140837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:34.201920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa204ab-b526-4a5a-a15d-eddaf3ce1f84 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:34.202297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa204ab-b526-4a5a-a15d-eddaf3ce1f84 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:34.202807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa204ab-b526-4a5a-a15d-eddaf3ce1f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:34.255141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2fa204ab-b526-4a5a-a15d-eddaf3ce1f84 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:34.259221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2fa204ab-b526-4a5a-a15d-eddaf3ce1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:34.421298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 719/1449] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:38:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:34.450553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7598284269a0410ba7872901531f8203'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:34.473629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7598284269a0410ba7872901531f8203'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:34.476544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7598284269a0410ba7872901531f8203'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:34.546210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7598284269a0410ba7872901531f8203'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:34.562047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7598284269a0410ba7872901531f8203'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:34.573869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 731/1450] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:33 2022] DELETE /identity/v3/users/7598284269a0410ba7872901531f8203 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:34.591880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:34.689910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa204ab-b526-4a5a-a15d-eddaf3ce1f84 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:34.697793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2fa204ab-b526-4a5a-a15d-eddaf3ce1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:34.715072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa204ab-b526-4a5a-a15d-eddaf3ce1f84 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:34.872580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:34.873298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:34.874295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:34.874784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:34.879901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:34.900320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:34.902299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:34.932190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 720/1451] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:35.800871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:35.932822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:35.934373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:36.102595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851ac43-8ab5-4fe5-9c15-36156dbebc57 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.107903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.108616 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.109052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.109524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] PATH_INFO: `/v3/projects/5b32af54526643ab9c9646fd3b61641c` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.112964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b32af54526643ab9c9646fd3b61641c)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:36.112964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:36.245572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b32af54526643ab9c9646fd3b61641c', 'request_id': 'req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.252763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b32af54526643ab9c9646fd3b61641c', 'request_id': 'req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.259739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b32af54526643ab9c9646fd3b61641c', 'request_id': 'req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.314075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:36.314075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:36.331795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7a401-c1db-4c96-a44f-50f50f190dd6 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.343669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] The token cache is being invalidate because project 5b32af54526643ab9c9646fd3b61641c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:36.345372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.347185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b6a8a8dd-e9ea-4f36-98c7-2ba27d1e8b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.352763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 732/1452] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:35 2022] DELETE /identity/v3/projects/5b32af54526643ab9c9646fd3b61641c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:36.358220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c1bd0-ca15-4e5a-b06c-e35182e9d139 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.359146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c1bd0-ca15-4e5a-b06c-e35182e9d139 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.359921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c1bd0-ca15-4e5a-b06c-e35182e9d139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.386033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-149c1bd0-ca15-4e5a-b06c-e35182e9d139 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:36.387739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-149c1bd0-ca15-4e5a-b06c-e35182e9d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:36.407854 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.408309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.408596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.408871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] PATH_INFO: `/v3/projects/82be8c6efc0e4fe388d5030d5884e38f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.410200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82be8c6efc0e4fe388d5030d5884e38f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:36.412128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:36.457854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 733/1453] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:36.465471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a8c9-fdfc-4158-a660-5c8f9ff19c8f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.465757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a8c9-fdfc-4158-a660-5c8f9ff19c8f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.466112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3a8c9-fdfc-4158-a660-5c8f9ff19c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.485543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-bbf3a8c9-fdfc-4158-a660-5c8f9ff19c8f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:36.487320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bbf3a8c9-fdfc-4158-a660-5c8f9ff19c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:36.495502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 734/1454] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:36.499551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3a8c9-fdfc-4158-a660-5c8f9ff19c8f None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:36.509248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3a8c9-fdfc-4158-a660-5c8f9ff19c8f None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.517812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82be8c6efc0e4fe388d5030d5884e38f', 'request_id': 'req-3d7142e9-d825-4bf8-a07c-abe27671e401'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.525114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82be8c6efc0e4fe388d5030d5884e38f', 'request_id': 'req-3d7142e9-d825-4bf8-a07c-abe27671e401'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.531481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82be8c6efc0e4fe388d5030d5884e38f', 'request_id': 'req-3d7142e9-d825-4bf8-a07c-abe27671e401'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.566269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.566872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.567150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.567420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.568051 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:36.568352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:36.569981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:36.585580 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 735/1455] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:38:36.589799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:36.601022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3065-9cad-4649-aec6-f3d676826fe0 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.606772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.607518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.607849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.608162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.608737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:36.610786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:36.614367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] The token cache is being invalidate because project 82be8c6efc0e4fe388d5030d5884e38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:36.614661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.615639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.617888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 721/1456] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:36 2022] DELETE /identity/v3/projects/82be8c6efc0e4fe388d5030d5884e38f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:36.621953 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:36.623484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:36.638071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7142e9-d825-4bf8-a07c-abe27671e401 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.648086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.648775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.649064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.649344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] PATH_INFO: `/v3/users/ce37c84b3e88462082a9080b2762edf3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.650608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce37c84b3e88462082a9080b2762edf3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:36.652771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:36.657896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:36.690684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 736/1457] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:36 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:38:36.691868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:36.693552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:36.760574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52484b30-6fac-4a79-9f5d-4c1778483847 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.765469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41eea8-5312-4877-b7c2-8ff2c8ddc42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.766120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41eea8-5312-4877-b7c2-8ff2c8ddc42d admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.766400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41eea8-5312-4877-b7c2-8ff2c8ddc42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.766674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41eea8-5312-4877-b7c2-8ff2c8ddc42d admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.767306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41eea8-5312-4877-b7c2-8ff2c8ddc42d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:36.769256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41eea8-5312-4877-b7c2-8ff2c8ddc42d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:36.877557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce37c84b3e88462082a9080b2762edf3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.894223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce37c84b3e88462082a9080b2762edf3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.895109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 737/1458] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:36 2022] POST /identity/v3/users => generated 294 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:36.898978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41eea8-5312-4877-b7c2-8ff2c8ddc42d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:36.910263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41eea8-5312-4877-b7c2-8ff2c8ddc42d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.915385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acffbf5-3972-4221-b4c0-c8c9e0fcc6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.916104 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acffbf5-3972-4221-b4c0-c8c9e0fcc6fe admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.919697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acffbf5-3972-4221-b4c0-c8c9e0fcc6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.919697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acffbf5-3972-4221-b4c0-c8c9e0fcc6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.919697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acffbf5-3972-4221-b4c0-c8c9e0fcc6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:36.919697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acffbf5-3972-4221-b4c0-c8c9e0fcc6fe admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:36.934069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 738/1459] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:36.935154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acffbf5-3972-4221-b4c0-c8c9e0fcc6fe admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:36.945248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acffbf5-3972-4221-b4c0-c8c9e0fcc6fe admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.950190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71425e4a-3698-4a81-a37c-acb77a6653d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.950779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71425e4a-3698-4a81-a37c-acb77a6653d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.951064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71425e4a-3698-4a81-a37c-acb77a6653d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.951336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71425e4a-3698-4a81-a37c-acb77a6653d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.951973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71425e4a-3698-4a81-a37c-acb77a6653d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:36.953593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71425e4a-3698-4a81-a37c-acb77a6653d6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:36.964832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 739/1460] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:36.968944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71425e4a-3698-4a81-a37c-acb77a6653d6 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:36.979535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71425e4a-3698-4a81-a37c-acb77a6653d6 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:36.984213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:36.984841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:36.985121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:36.985397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] PATH_INFO: `/v3/projects/2318e68b466a4125a22f17c3bab8216d/users/b53bcfc8e6864f1aad17bc8460017278/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:36.996380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce37c84b3e88462082a9080b2762edf3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:36.996974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce37c84b3e88462082a9080b2762edf3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:37.000661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2318e68b466a4125a22f17c3bab8216d, user_id=b53bcfc8e6864f1aad17bc8460017278, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.003624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.004610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60480598-68d7-4e0e-9de5-06bcc1cbdce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce37c84b3e88462082a9080b2762edf3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:37.011228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 722/1461] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:36 2022] DELETE /identity/v3/users/ce37c84b3e88462082a9080b2762edf3 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:37.101275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 740/1462] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:36 2022] PUT /identity/v3/projects/2318e68b466a4125a22f17c3bab8216d/users/b53bcfc8e6864f1aad17bc8460017278/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:37.111740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f258b-b180-41c2-ba9e-01019d69f24e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.112229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f258b-b180-41c2-ba9e-01019d69f24e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.112600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f258b-b180-41c2-ba9e-01019d69f24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.141789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b99f258b-b180-41c2-ba9e-01019d69f24e None None] MFA Rules not processed for user `b53bcfc8e6864f1aad17bc8460017278`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:37.143618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b99f258b-b180-41c2-ba9e-01019d69f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:37.211057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 723/1463] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:37.215738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.217546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:37.286465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e456c-809f-444c-bf62-5d2ece0918b5 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.291439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.293850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.293850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.293850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.293850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.295454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.415104 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 741/1464] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:37 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:37.420157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f258b-b180-41c2-ba9e-01019d69f24e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.431946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f258b-b180-41c2-ba9e-01019d69f24e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.437107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.439113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:37.498980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4f29b-bfa7-4ccb-b25f-7f2aee2b8228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.499617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4f29b-bfa7-4ccb-b25f-7f2aee2b8228 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.499961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4f29b-bfa7-4ccb-b25f-7f2aee2b8228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.500299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4f29b-bfa7-4ccb-b25f-7f2aee2b8228 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.501019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4f29b-bfa7-4ccb-b25f-7f2aee2b8228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.503153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da4f29b-bfa7-4ccb-b25f-7f2aee2b8228 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.515061 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21658046-1a43-4530-af1d-60a75566487a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.520000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.520668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.520952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.521225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] PATH_INFO: `/v3/projects/bf34f926952f4b64bb888efcecadb394` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.522397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf34f926952f4b64bb888efcecadb394)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.524295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.543112 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:37.597977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 724/1465] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:37 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:37.602002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4f29b-bfa7-4ccb-b25f-7f2aee2b8228 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.616086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4f29b-bfa7-4ccb-b25f-7f2aee2b8228 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.620931 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859eb6e-057e-4dec-9aae-4f08b09c2337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.621537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859eb6e-057e-4dec-9aae-4f08b09c2337 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.621816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859eb6e-057e-4dec-9aae-4f08b09c2337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.622111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859eb6e-057e-4dec-9aae-4f08b09c2337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.622755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859eb6e-057e-4dec-9aae-4f08b09c2337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.624647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859eb6e-057e-4dec-9aae-4f08b09c2337 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.638012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 725/1466] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:37.641144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859eb6e-057e-4dec-9aae-4f08b09c2337 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.645928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf34f926952f4b64bb888efcecadb394', 'request_id': 'req-6890cc4c-75e6-4fb0-8289-ef500171c7a7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:37.651745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf34f926952f4b64bb888efcecadb394', 'request_id': 'req-6890cc4c-75e6-4fb0-8289-ef500171c7a7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:37.652204 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859eb6e-057e-4dec-9aae-4f08b09c2337 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.658181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf34f926952f4b64bb888efcecadb394', 'request_id': 'req-6890cc4c-75e6-4fb0-8289-ef500171c7a7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:37.707700 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.708344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.708630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.708904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] PATH_INFO: `/v3/projects/5f2aab3230404a1b99650e5ebf793807/users/d1c9786757e14aeb90eeb622ba623caa/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.722980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2aab3230404a1b99650e5ebf793807, user_id=d1c9786757e14aeb90eeb622ba623caa, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.725603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.750185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] The token cache is being invalidate because project bf34f926952f4b64bb888efcecadb394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:37.750424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:37.751073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6890cc4c-75e6-4fb0-8289-ef500171c7a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:37.752626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 742/1467] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:37 2022] DELETE /identity/v3/projects/bf34f926952f4b64bb888efcecadb394 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:37.757371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c7109-a61e-4846-8b7b-0c616f382c2f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.757651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c7109-a61e-4846-8b7b-0c616f382c2f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.757980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c7109-a61e-4846-8b7b-0c616f382c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.777769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-267c7109-a61e-4846-8b7b-0c616f382c2f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:37.779057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-267c7109-a61e-4846-8b7b-0c616f382c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:37.787762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 743/1468] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:37.795195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c284c-6c8f-43b3-aff2-c110e1a69e5d None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.795469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c284c-6c8f-43b3-aff2-c110e1a69e5d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.795736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c284c-6c8f-43b3-aff2-c110e1a69e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.814712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e55c284c-6c8f-43b3-aff2-c110e1a69e5d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:37.815881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e55c284c-6c8f-43b3-aff2-c110e1a69e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:37.823601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 744/1469] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:37.826814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 726/1470] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:37 2022] PUT /identity/v3/projects/5f2aab3230404a1b99650e5ebf793807/users/d1c9786757e14aeb90eeb622ba623caa/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:37.828092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c284c-6c8f-43b3-aff2-c110e1a69e5d None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.829117 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.831553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:37.838034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c284c-6c8f-43b3-aff2-c110e1a69e5d None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.894161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.895026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.895341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425033a-59d0-4986-8d2a-7c1b183fafc1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.895803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.895803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.896660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:37.897037 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.900070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.900270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44377fd-4ded-450b-91ed-b07508b37c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.900874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44377fd-4ded-450b-91ed-b07508b37c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.901158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44377fd-4ded-450b-91ed-b07508b37c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.901437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44377fd-4ded-450b-91ed-b07508b37c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.902115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44377fd-4ded-450b-91ed-b07508b37c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.903724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44377fd-4ded-450b-91ed-b07508b37c1b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.912744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 745/1471] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:38:37.914559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 727/1472] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:37.918310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.918760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44377fd-4ded-450b-91ed-b07508b37c1b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:37.928345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd845c9-8edf-48ea-9e74-d31ffc6e7259 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.928777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44377fd-4ded-450b-91ed-b07508b37c1b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:37.932994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805c92c-9703-4959-a302-17737ca5b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.933350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670225cc-f479-4204-a20a-1353c383bf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:37.933590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c92c-9703-4959-a302-17737ca5b973 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.933888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c92c-9703-4959-a302-17737ca5b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.934033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670225cc-f479-4204-a20a-1353c383bf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.934191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c92c-9703-4959-a302-17737ca5b973 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.934600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670225cc-f479-4204-a20a-1353c383bf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.934600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670225cc-f479-4204-a20a-1353c383bf34 admin admin] PATH_INFO: `/v3/projects/5f2aab3230404a1b99650e5ebf793807/users/d1c9786757e14aeb90eeb622ba623caa/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.934834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805c92c-9703-4959-a302-17737ca5b973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.936431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670225cc-f479-4204-a20a-1353c383bf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2aab3230404a1b99650e5ebf793807, user_id=d1c9786757e14aeb90eeb622ba623caa, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:37.936779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805c92c-9703-4959-a302-17737ca5b973 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.939143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670225cc-f479-4204-a20a-1353c383bf34 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:37.957511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 728/1473] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:37 2022] PUT /identity/v3/projects/5f2aab3230404a1b99650e5ebf793807/users/d1c9786757e14aeb90eeb622ba623caa/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:37.964375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc0c8-e4c0-4ec1-ad14-844ebfe36825 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:37.964656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc0c8-e4c0-4ec1-ad14-844ebfe36825 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:37.964930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc0c8-e4c0-4ec1-ad14-844ebfe36825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:37.994750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-637cc0c8-e4c0-4ec1-ad14-844ebfe36825 None None] MFA Rules not processed for user `d1c9786757e14aeb90eeb622ba623caa`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:37.996006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-637cc0c8-e4c0-4ec1-ad14-844ebfe36825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:38.072886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 729/1474] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:37 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:38.082947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cc0c8-e4c0-4ec1-ad14-844ebfe36825 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.084695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-637cc0c8-e4c0-4ec1-ad14-844ebfe36825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:38.085335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 746/1475] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:37 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:38:38.089208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805c92c-9703-4959-a302-17737ca5b973 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.144558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cc0c8-e4c0-4ec1-ad14-844ebfe36825 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:38.153488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805c92c-9703-4959-a302-17737ca5b973 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:38.158252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce899f22-1621-4337-aa6d-a4d443f0235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:38.158941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899f22-1621-4337-aa6d-a4d443f0235b admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.159218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899f22-1621-4337-aa6d-a4d443f0235b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.159487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce899f22-1621-4337-aa6d-a4d443f0235b admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.160109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce899f22-1621-4337-aa6d-a4d443f0235b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:38.162050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce899f22-1621-4337-aa6d-a4d443f0235b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:38.167515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:38.192670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42d2e3-f80c-474f-aa6d-68a749b2234d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:38.193347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42d2e3-f80c-474f-aa6d-68a749b2234d service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.193625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42d2e3-f80c-474f-aa6d-68a749b2234d service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.193895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42d2e3-f80c-474f-aa6d-68a749b2234d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.255666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42d2e3-f80c-474f-aa6d-68a749b2234d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:38.257343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42d2e3-f80c-474f-aa6d-68a749b2234d service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:38.259521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 747/1476] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:38 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:38.263475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce899f22-1621-4337-aa6d-a4d443f0235b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.271853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 730/1477] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:38 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:38.276343 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce899f22-1621-4337-aa6d-a4d443f0235b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:38.289112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d551e9-8be9-4524-a37a-24cc1902f18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:38.289112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d551e9-8be9-4524-a37a-24cc1902f18d admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.289112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d551e9-8be9-4524-a37a-24cc1902f18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.289112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d551e9-8be9-4524-a37a-24cc1902f18d admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.290154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d551e9-8be9-4524-a37a-24cc1902f18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:38.290154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d551e9-8be9-4524-a37a-24cc1902f18d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:38.300060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 748/1478] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:38.303999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42d2e3-f80c-474f-aa6d-68a749b2234d service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.319925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42d2e3-f80c-474f-aa6d-68a749b2234d service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:38.325139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b4e27-4c2d-47b8-82bb-917f1f1b2161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:38.326066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4e27-4c2d-47b8-82bb-917f1f1b2161 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.326447 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4e27-4c2d-47b8-82bb-917f1f1b2161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.326817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b4e27-4c2d-47b8-82bb-917f1f1b2161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.328278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b4e27-4c2d-47b8-82bb-917f1f1b2161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:38.330144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b4e27-4c2d-47b8-82bb-917f1f1b2161 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:38.348930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 731/1479] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:38.350417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d551e9-8be9-4524-a37a-24cc1902f18d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.361110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d551e9-8be9-4524-a37a-24cc1902f18d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:38.366541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e62dc-a7d4-45a6-8039-96aa7266b987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:38.367311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e62dc-a7d4-45a6-8039-96aa7266b987 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.367657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e62dc-a7d4-45a6-8039-96aa7266b987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.368009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e62dc-a7d4-45a6-8039-96aa7266b987 admin admin] PATH_INFO: `/v3/projects/5f76fcdcde7c4cf1a5bf1e020e8b38fd/users/f016c1c031ff42eda16dddc301945620/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.393452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e62dc-a7d4-45a6-8039-96aa7266b987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f76fcdcde7c4cf1a5bf1e020e8b38fd, user_id=f016c1c031ff42eda16dddc301945620, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:38.396598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e62dc-a7d4-45a6-8039-96aa7266b987 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:38.429709 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b4e27-4c2d-47b8-82bb-917f1f1b2161 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.443245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b4e27-4c2d-47b8-82bb-917f1f1b2161 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:38.448618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:38.449444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.449794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.450202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] PATH_INFO: `/v3/users/b53bcfc8e6864f1aad17bc8460017278` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.451644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b53bcfc8e6864f1aad17bc8460017278)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:38.453645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:38.523356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 749/1480] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:38 2022] PUT /identity/v3/projects/5f76fcdcde7c4cf1a5bf1e020e8b38fd/users/f016c1c031ff42eda16dddc301945620/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:38.534237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175521e6-a03c-4660-b67e-e898886d85c6 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.534523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175521e6-a03c-4660-b67e-e898886d85c6 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.534800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175521e6-a03c-4660-b67e-e898886d85c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.571162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-175521e6-a03c-4660-b67e-e898886d85c6 None None] MFA Rules not processed for user `f016c1c031ff42eda16dddc301945620`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:38.572375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-175521e6-a03c-4660-b67e-e898886d85c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:38.704309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 750/1481] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:38.718954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175521e6-a03c-4660-b67e-e898886d85c6 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.761426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b53bcfc8e6864f1aad17bc8460017278'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:38.787159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b53bcfc8e6864f1aad17bc8460017278'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:38.832408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175521e6-a03c-4660-b67e-e898886d85c6 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:38.843032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:38.843032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.843032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.843032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.843577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b53bcfc8e6864f1aad17bc8460017278'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:38.843577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b53bcfc8e6864f1aad17bc8460017278'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:38.845544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:38.848472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:38.851345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9c4afa58-9ac4-418c-99b0-8f5aa025b6f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b53bcfc8e6864f1aad17bc8460017278'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:38.858193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 732/1482] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:38 2022] DELETE /identity/v3/users/b53bcfc8e6864f1aad17bc8460017278 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:38.863543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9239735-fea1-465f-bee4-ef31fb7859f7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:38.863938 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9239735-fea1-465f-bee4-ef31fb7859f7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:38.864354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9239735-fea1-465f-bee4-ef31fb7859f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:38.887099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a9239735-fea1-465f-bee4-ef31fb7859f7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:38.888400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a9239735-fea1-465f-bee4-ef31fb7859f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:38.940780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 751/1483] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:38 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:38.944947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.946713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:38.962450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 733/1484] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:38.973549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9239735-fea1-465f-bee4-ef31fb7859f7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:38.974966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a9239735-fea1-465f-bee4-ef31fb7859f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:39.037924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cd29f-1f39-49c9-b39f-e0525ef2d3b7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:39.044509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e663e0-3d55-48ce-be8a-3e36fea76751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:39.045136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e663e0-3d55-48ce-be8a-3e36fea76751 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:39.045418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e663e0-3d55-48ce-be8a-3e36fea76751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:39.045694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e663e0-3d55-48ce-be8a-3e36fea76751 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:39.046717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e663e0-3d55-48ce-be8a-3e36fea76751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:39.048783 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e663e0-3d55-48ce-be8a-3e36fea76751 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:39.056285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9239735-fea1-465f-bee4-ef31fb7859f7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:39.120889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0d3a9-0d8d-4337-9b49-82b624b9ed1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:39.121421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d3a9-0d8d-4337-9b49-82b624b9ed1e service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:39.121845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d3a9-0d8d-4337-9b49-82b624b9ed1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:39.122166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0d3a9-0d8d-4337-9b49-82b624b9ed1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:39.133682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 752/1485] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:38 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:39.142026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e663e0-3d55-48ce-be8a-3e36fea76751 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:39.142131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0d3a9-0d8d-4337-9b49-82b624b9ed1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:39.142131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0d3a9-0d8d-4337-9b49-82b624b9ed1e service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:39.150437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e663e0-3d55-48ce-be8a-3e36fea76751 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:39.156021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122da543-404f-4a24-ad7b-ea1cc46c7625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:39.157000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 734/1486] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:39.157125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122da543-404f-4a24-ad7b-ea1cc46c7625 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:39.157593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122da543-404f-4a24-ad7b-ea1cc46c7625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:39.158034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122da543-404f-4a24-ad7b-ea1cc46c7625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:39.159446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122da543-404f-4a24-ad7b-ea1cc46c7625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:39.161442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122da543-404f-4a24-ad7b-ea1cc46c7625 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:39.174459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 753/1487] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:39.178644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0d3a9-0d8d-4337-9b49-82b624b9ed1e service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:39.190961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0d3a9-0d8d-4337-9b49-82b624b9ed1e service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:39.197515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa640292-34f5-4343-abca-5d3343f36c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:39.199817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa640292-34f5-4343-abca-5d3343f36c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:39.199817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa640292-34f5-4343-abca-5d3343f36c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:39.199817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa640292-34f5-4343-abca-5d3343f36c6a admin admin] PATH_INFO: `/v3/projects/f4e069cf5bc84c01ba5171aec8179aa9/users/e6736005de0e493cb40f1dd9d59a2ca5/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:39.233355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa640292-34f5-4343-abca-5d3343f36c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e069cf5bc84c01ba5171aec8179aa9, user_id=e6736005de0e493cb40f1dd9d59a2ca5, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:39.236168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa640292-34f5-4343-abca-5d3343f36c6a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:39.340957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 735/1488] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:39 2022] PUT /identity/v3/projects/f4e069cf5bc84c01ba5171aec8179aa9/users/e6736005de0e493cb40f1dd9d59a2ca5/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:39.351371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122da543-404f-4a24-ad7b-ea1cc46c7625 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:39.362865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:39.366755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122da543-404f-4a24-ad7b-ea1cc46c7625 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:39.425755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:39.460691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cd58b-5a85-4460-b164-3fe2d9d74aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:39.461331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cd58b-5a85-4460-b164-3fe2d9d74aba admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:39.461615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cd58b-5a85-4460-b164-3fe2d9d74aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:39.462093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cd58b-5a85-4460-b164-3fe2d9d74aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:39.462819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cd58b-5a85-4460-b164-3fe2d9d74aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:39.464410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cd58b-5a85-4460-b164-3fe2d9d74aba admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:39.480128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 754/1489] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:39.481193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa640292-34f5-4343-abca-5d3343f36c6a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:39.494313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa640292-34f5-4343-abca-5d3343f36c6a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:39.500620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:39.501344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:39.501627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:39.501900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] PATH_INFO: `/v3/projects/f4e069cf5bc84c01ba5171aec8179aa9/users/e6736005de0e493cb40f1dd9d59a2ca5/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:39.504820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e069cf5bc84c01ba5171aec8179aa9, user_id=e6736005de0e493cb40f1dd9d59a2ca5, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:39.508334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:39.546226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 736/1490] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:39 2022] PUT /identity/v3/projects/f4e069cf5bc84c01ba5171aec8179aa9/users/e6736005de0e493cb40f1dd9d59a2ca5/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:39.550424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e88a6-4f0e-4b32-abdc-d90cfdc96950 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:39.550882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e88a6-4f0e-4b32-abdc-d90cfdc96950 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:39.551346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e88a6-4f0e-4b32-abdc-d90cfdc96950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:39.624550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-168e88a6-4f0e-4b32-abdc-d90cfdc96950 None None] MFA Rules not processed for user `e6736005de0e493cb40f1dd9d59a2ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:39.629742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-168e88a6-4f0e-4b32-abdc-d90cfdc96950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:39.746218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:39.747308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:39.820928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 755/1491] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:39.835863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:39.846552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e88a6-4f0e-4b32-abdc-d90cfdc96950 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:39.848283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-168e88a6-4f0e-4b32-abdc-d90cfdc96950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:39.927110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c6be7-df29-46c8-9aee-b80ec18ca33c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:39.934829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:39.936576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:39.937095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:39.937571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] PATH_INFO: `/v3/projects/2318e68b466a4125a22f17c3bab8216d` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:39.939015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=2318e68b466a4125a22f17c3bab8216d)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:39.941414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:39.994014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e88a6-4f0e-4b32-abdc-d90cfdc96950 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:39.998067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cb099-9537-458b-878d-c7f7116a5f0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:40.000011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cb099-9537-458b-878d-c7f7116a5f0d service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:40.000011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cb099-9537-458b-878d-c7f7116a5f0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:40.000011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cb099-9537-458b-878d-c7f7116a5f0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:40.009302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cb099-9537-458b-878d-c7f7116a5f0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:40.010787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cb099-9537-458b-878d-c7f7116a5f0d service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:40.037621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 756/1492] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:40.089839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2318e68b466a4125a22f17c3bab8216d', 'request_id': 'req-ebbbfa57-e2a3-432f-b877-11099f664dad'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.100426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2318e68b466a4125a22f17c3bab8216d', 'request_id': 'req-ebbbfa57-e2a3-432f-b877-11099f664dad'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.102921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2318e68b466a4125a22f17c3bab8216d', 'request_id': 'req-ebbbfa57-e2a3-432f-b877-11099f664dad'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.202665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] The token cache is being invalidate because project 2318e68b466a4125a22f17c3bab8216d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:40.206197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.206197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.206197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 737/1493] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:39 2022] DELETE /identity/v3/projects/2318e68b466a4125a22f17c3bab8216d => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:40.209512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cb099-9537-458b-878d-c7f7116a5f0d service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:40.220189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cb099-9537-458b-878d-c7f7116a5f0d service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:40.257001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:40.284033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:40.286733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:40.286733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:40.286733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] PATH_INFO: `/v3/users/d1c9786757e14aeb90eeb622ba623caa` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:40.287290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c9786757e14aeb90eeb622ba623caa)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:40.289227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:40.523546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c9786757e14aeb90eeb622ba623caa'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.538155 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c9786757e14aeb90eeb622ba623caa'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.538936 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c9786757e14aeb90eeb622ba623caa'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.649043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c9786757e14aeb90eeb622ba623caa'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.656676 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c9786757e14aeb90eeb622ba623caa'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:40.680215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 757/1494] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:40 2022] DELETE /identity/v3/users/d1c9786757e14aeb90eeb622ba623caa => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:41.098906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:41.180061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:41.342734 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:41.345197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:41.364505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbfa57-e2a3-432f-b877-11099f664dad admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:41.425269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:41.425911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:41.426223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:41.426496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] PATH_INFO: `/v3/projects/5f2aab3230404a1b99650e5ebf793807` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:41.427647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2aab3230404a1b99650e5ebf793807)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:41.429665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:41.480153 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:41.481555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:41.494772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9d317-b0aa-463e-b139-03041504c922 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:41.499488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:41.500110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:41.500412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:41.500684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] PATH_INFO: `/v3/users/f016c1c031ff42eda16dddc301945620` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:41.501791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f016c1c031ff42eda16dddc301945620)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:41.503765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:41.535042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2aab3230404a1b99650e5ebf793807', 'request_id': 'req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.541398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2aab3230404a1b99650e5ebf793807', 'request_id': 'req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.547541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2aab3230404a1b99650e5ebf793807', 'request_id': 'req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.624782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] The token cache is being invalidate because project 5f2aab3230404a1b99650e5ebf793807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:41.625075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.625718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-68b9ebbe-5a0e-4241-9fc9-2067b476a4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.627297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 738/1495] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:41 2022] DELETE /identity/v3/projects/5f2aab3230404a1b99650e5ebf793807 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:41.631896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e8b6-426f-47f6-9753-63b15c8247a5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:41.632186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e8b6-426f-47f6-9753-63b15c8247a5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:41.632462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e8b6-426f-47f6-9753-63b15c8247a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:41.653094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c945e8b6-426f-47f6-9753-63b15c8247a5 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:41.654355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c945e8b6-426f-47f6-9753-63b15c8247a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:41.665048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f016c1c031ff42eda16dddc301945620'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.680665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f016c1c031ff42eda16dddc301945620'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.681395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f016c1c031ff42eda16dddc301945620'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.720689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 739/1496] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:41.728155 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f016c1c031ff42eda16dddc301945620'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.728845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4e88b-e711-4c98-8611-8af8136390e6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:41.729134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4e88b-e711-4c98-8611-8af8136390e6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:41.729415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4e88b-e711-4c98-8611-8af8136390e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:41.735833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ba36ad5f-e6a0-44e9-979f-6817332a924e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f016c1c031ff42eda16dddc301945620'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:41.742174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 758/1497] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:41 2022] DELETE /identity/v3/users/f016c1c031ff42eda16dddc301945620 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:41.747866 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaf17d-07fe-427c-80b4-f8b9261633a4 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:41.748150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaf17d-07fe-427c-80b4-f8b9261633a4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:41.748425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaf17d-07fe-427c-80b4-f8b9261633a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:41.750561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-cea4e88b-e711-4c98-8611-8af8136390e6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:41.751831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-cea4e88b-e711-4c98-8611-8af8136390e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:41.769488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e0eaf17d-07fe-427c-80b4-f8b9261633a4 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:41.770857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e0eaf17d-07fe-427c-80b4-f8b9261633a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:41.810216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 740/1498] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:41.814450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4e88b-e711-4c98-8611-8af8136390e6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:41.824189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4e88b-e711-4c98-8611-8af8136390e6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:41.828959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:41.829582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:41.829861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:41.830170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:41.831415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:41.831721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:41.833317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:41.833811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 759/1499] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:41.843408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaf17d-07fe-427c-80b4-f8b9261633a4 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:41.844858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e0eaf17d-07fe-427c-80b4-f8b9261633a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:41.848694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 741/1500] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:38:41.852706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:41.863578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d5c5d-b358-492a-9aa5-7113be8afa84 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:41.868483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c9d6f-162d-4177-a3ae-82b9678dfdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:41.869189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c9d6f-162d-4177-a3ae-82b9678dfdde admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:41.869471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c9d6f-162d-4177-a3ae-82b9678dfdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:41.869767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c9d6f-162d-4177-a3ae-82b9678dfdde admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:41.870437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c9d6f-162d-4177-a3ae-82b9678dfdde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:41.872321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c9d6f-162d-4177-a3ae-82b9678dfdde admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:41.886818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:41.908480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaf17d-07fe-427c-80b4-f8b9261633a4 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:41.909673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:41.912091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d6425-bccc-44b9-aa4c-43029526c31e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:41.912691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d6425-bccc-44b9-aa4c-43029526c31e service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:41.913006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d6425-bccc-44b9-aa4c-43029526c31e service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:41.913276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d6425-bccc-44b9-aa4c-43029526c31e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:41.922324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d6425-bccc-44b9-aa4c-43029526c31e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:41.923773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d6425-bccc-44b9-aa4c-43029526c31e service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:41.933208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 760/1501] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:42.000191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 742/1502] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:41 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:38:42.001752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d6425-bccc-44b9-aa4c-43029526c31e service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:42.016345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d6425-bccc-44b9-aa4c-43029526c31e service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:42.084493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168e6ef-2f45-4bad-916b-c7722dd849b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:42.084493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168e6ef-2f45-4bad-916b-c7722dd849b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:42.084493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168e6ef-2f45-4bad-916b-c7722dd849b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:42.085273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168e6ef-2f45-4bad-916b-c7722dd849b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:42.085273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168e6ef-2f45-4bad-916b-c7722dd849b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:42.087343 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168e6ef-2f45-4bad-916b-c7722dd849b8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:42.199073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 761/1503] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:42 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:42.202722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c9d6f-162d-4177-a3ae-82b9678dfdde admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:42.218583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c9d6f-162d-4177-a3ae-82b9678dfdde admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:42.225553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30946fc3-8cd9-4ab3-be54-63f7ca2bedfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:42.230055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30946fc3-8cd9-4ab3-be54-63f7ca2bedfd admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:42.230055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30946fc3-8cd9-4ab3-be54-63f7ca2bedfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:42.230055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30946fc3-8cd9-4ab3-be54-63f7ca2bedfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:42.230055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30946fc3-8cd9-4ab3-be54-63f7ca2bedfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:42.233572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30946fc3-8cd9-4ab3-be54-63f7ca2bedfd admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:42.259865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 743/1504] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:42.264514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168e6ef-2f45-4bad-916b-c7722dd849b8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:42.288765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168e6ef-2f45-4bad-916b-c7722dd849b8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:42.303216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306ce2a-9a80-4f23-a849-f757be64c58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:42.304424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306ce2a-9a80-4f23-a849-f757be64c58b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:42.304933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306ce2a-9a80-4f23-a849-f757be64c58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:42.305406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306ce2a-9a80-4f23-a849-f757be64c58b admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:42.306383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306ce2a-9a80-4f23-a849-f757be64c58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:42.309955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306ce2a-9a80-4f23-a849-f757be64c58b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:42.341772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 762/1505] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:42.349313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30946fc3-8cd9-4ab3-be54-63f7ca2bedfd admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:42.376128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30946fc3-8cd9-4ab3-be54-63f7ca2bedfd admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:42.383454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:42.383971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:42.384381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:42.384758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] PATH_INFO: `/v3/projects/b3a15af4aaf24498bdcfea58d8d23b20/users/2930fa84635642ccb637e38019984ee4/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:42.408809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a15af4aaf24498bdcfea58d8d23b20, user_id=2930fa84635642ccb637e38019984ee4, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:42.414364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:42.504006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 744/1506] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:42 2022] PUT /identity/v3/projects/b3a15af4aaf24498bdcfea58d8d23b20/users/2930fa84635642ccb637e38019984ee4/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:42.514416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b21d08-a85c-4101-8c29-b52ad093e314 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:42.514834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b21d08-a85c-4101-8c29-b52ad093e314 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:42.515207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b21d08-a85c-4101-8c29-b52ad093e314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:42.575213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c0b21d08-a85c-4101-8c29-b52ad093e314 None None] MFA Rules not processed for user `2930fa84635642ccb637e38019984ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:42.576762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c0b21d08-a85c-4101-8c29-b52ad093e314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:42.603428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:42.605311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:42.781694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 763/1507] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:42.789725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4c26f-920e-456b-b6c9-6ae2cc82cd33 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:42.789872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b21d08-a85c-4101-8c29-b52ad093e314 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:42.806662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:42.808873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:42.810311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:42.810675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] PATH_INFO: `/v3/projects/5f76fcdcde7c4cf1a5bf1e020e8b38fd` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:42.811559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b21d08-a85c-4101-8c29-b52ad093e314 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:42.812615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f76fcdcde7c4cf1a5bf1e020e8b38fd)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:42.816647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:42.823523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44d029-02bf-475c-8a60-80201d8bfd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:42.825473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44d029-02bf-475c-8a60-80201d8bfd55 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:42.825972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44d029-02bf-475c-8a60-80201d8bfd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:42.826798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44d029-02bf-475c-8a60-80201d8bfd55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:42.827661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44d029-02bf-475c-8a60-80201d8bfd55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:42.833007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44d029-02bf-475c-8a60-80201d8bfd55 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:42.870761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:42.973135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 764/1508] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:42 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:38:42.977437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44d029-02bf-475c-8a60-80201d8bfd55 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:42.988109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:43.028994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f76fcdcde7c4cf1a5bf1e020e8b38fd', 'request_id': 'req-82c9df83-2aca-4a43-b8b2-fdd3b954726c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.046936 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f76fcdcde7c4cf1a5bf1e020e8b38fd', 'request_id': 'req-82c9df83-2aca-4a43-b8b2-fdd3b954726c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.068793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f76fcdcde7c4cf1a5bf1e020e8b38fd', 'request_id': 'req-82c9df83-2aca-4a43-b8b2-fdd3b954726c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.126283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44d029-02bf-475c-8a60-80201d8bfd55 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:43.169333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] The token cache is being invalidate because project 5f76fcdcde7c4cf1a5bf1e020e8b38fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:43.169636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.170379 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.177296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 745/1509] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:42 2022] DELETE /identity/v3/projects/5f76fcdcde7c4cf1a5bf1e020e8b38fd => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:43.181169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:43.184992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:43.307600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:43.316075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:43.316075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:43.316075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:43.316075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:43.316075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:43.336027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9df83-2aca-4a43-b8b2-fdd3b954726c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:43.342116 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:43.342793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:43.345503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:43.345503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] PATH_INFO: `/v3/users/e6736005de0e493cb40f1dd9d59a2ca5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:43.345503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6736005de0e493cb40f1dd9d59a2ca5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:43.347851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:43.395997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 765/1510] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:42 2022] POST /identity/v3/users => generated 297 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:43.400164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:43.402268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:43.425547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b74303-b28a-470b-87b3-4dcc8d5c0c4c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:43.431557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163da28-1ebf-4b36-a2e9-2a4399830c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:43.432512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163da28-1ebf-4b36-a2e9-2a4399830c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:43.432818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163da28-1ebf-4b36-a2e9-2a4399830c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:43.433915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163da28-1ebf-4b36-a2e9-2a4399830c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:43.434643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163da28-1ebf-4b36-a2e9-2a4399830c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:43.437015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163da28-1ebf-4b36-a2e9-2a4399830c4a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:43.450790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 766/1511] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:43.455102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163da28-1ebf-4b36-a2e9-2a4399830c4a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:43.466650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163da28-1ebf-4b36-a2e9-2a4399830c4a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:43.472292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46b827-1924-4232-af75-a641077fd552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:43.473428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46b827-1924-4232-af75-a641077fd552 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:43.473909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46b827-1924-4232-af75-a641077fd552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:43.474509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46b827-1924-4232-af75-a641077fd552 admin admin] PATH_INFO: `/v3/projects/d64ca8c11c1c4180b40d11d013a0f26c/users/c58a2af6f3b1493a81baa37295dab6f4/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:43.496419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46b827-1924-4232-af75-a641077fd552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64ca8c11c1c4180b40d11d013a0f26c, user_id=c58a2af6f3b1493a81baa37295dab6f4, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:43.500483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46b827-1924-4232-af75-a641077fd552 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:43.586467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6736005de0e493cb40f1dd9d59a2ca5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.598529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 767/1512] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:43 2022] PUT /identity/v3/projects/d64ca8c11c1c4180b40d11d013a0f26c/users/c58a2af6f3b1493a81baa37295dab6f4/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:43.602275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46b827-1924-4232-af75-a641077fd552 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:43.624161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6736005de0e493cb40f1dd9d59a2ca5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.689124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46b827-1924-4232-af75-a641077fd552 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:43.694316 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a1e2d-a8c8-4f70-a728-357d4ce5535d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:43.695233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a1e2d-a8c8-4f70-a728-357d4ce5535d admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:43.695617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a1e2d-a8c8-4f70-a728-357d4ce5535d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:43.696102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a1e2d-a8c8-4f70-a728-357d4ce5535d admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:43.696851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a1e2d-a8c8-4f70-a728-357d4ce5535d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:43.698954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a1e2d-a8c8-4f70-a728-357d4ce5535d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:43.709894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 768/1513] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:43.714825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a1e2d-a8c8-4f70-a728-357d4ce5535d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:43.725826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a1e2d-a8c8-4f70-a728-357d4ce5535d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:43.728272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6736005de0e493cb40f1dd9d59a2ca5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.728966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6736005de0e493cb40f1dd9d59a2ca5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.736600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:43.738229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:43.739010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:43.739886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] PATH_INFO: `/v3/projects/d64ca8c11c1c4180b40d11d013a0f26c/users/c58a2af6f3b1493a81baa37295dab6f4/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:43.741350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-34f467df-f7fd-4521-b536-26140da477ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6736005de0e493cb40f1dd9d59a2ca5'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:43.743637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64ca8c11c1c4180b40d11d013a0f26c, user_id=c58a2af6f3b1493a81baa37295dab6f4, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:43.748380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:43.757781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 746/1514] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:43 2022] DELETE /identity/v3/users/e6736005de0e493cb40f1dd9d59a2ca5 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:43.775844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 769/1515] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:43 2022] PUT /identity/v3/projects/d64ca8c11c1c4180b40d11d013a0f26c/users/c58a2af6f3b1493a81baa37295dab6f4/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:43.778289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363306b8-c2b1-4845-9731-c122e86222f0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:43.778745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363306b8-c2b1-4845-9731-c122e86222f0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:43.779021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363306b8-c2b1-4845-9731-c122e86222f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:43.810680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-363306b8-c2b1-4845-9731-c122e86222f0 None None] MFA Rules not processed for user `c58a2af6f3b1493a81baa37295dab6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:43.811871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-363306b8-c2b1-4845-9731-c122e86222f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:43.889167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 747/1516] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:43.901652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:43.903389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:43.990349 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc43014-f845-476f-a047-49fcbc7c8cb4 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:43.994519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:43.995442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:43.995551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:43.995958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:44.008146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:44.009724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:44.023284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 770/1517] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:44.226254 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363306b8-c2b1-4845-9731-c122e86222f0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:44.228022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-363306b8-c2b1-4845-9731-c122e86222f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:44.241383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363306b8-c2b1-4845-9731-c122e86222f0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:44.245626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:44.246378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:44.246566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:44.246973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:44.248926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:44.330177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:44.330177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:44.359652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 748/1518] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:44.365193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:44.369016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:44.473157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46477dbd-b1b3-40d1-8638-7f3b9bd58100 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:44.478318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:44.479775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:44.480654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:44.480995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:44.481278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] PATH_INFO: `/v3/projects/f4e069cf5bc84c01ba5171aec8179aa9` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:44.482649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e069cf5bc84c01ba5171aec8179aa9)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:44.484619 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:44.554413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:44.557292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:44.573062 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b8b3f-038f-41e5-b066-5f4146e68710 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:44.577787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:44.578484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:44.578762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:44.579037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] PATH_INFO: `/v3/users/f5a6310201614d09939c820f49a5f2cb` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:44.581005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a6310201614d09939c820f49a5f2cb)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:44.582982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:44.589386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e069cf5bc84c01ba5171aec8179aa9', 'request_id': 'req-a46ff475-5ce3-411d-8c4c-fde825e0503a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:44.595393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e069cf5bc84c01ba5171aec8179aa9', 'request_id': 'req-a46ff475-5ce3-411d-8c4c-fde825e0503a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:44.604584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e069cf5bc84c01ba5171aec8179aa9', 'request_id': 'req-a46ff475-5ce3-411d-8c4c-fde825e0503a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:44.697348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] The token cache is being invalidate because project f4e069cf5bc84c01ba5171aec8179aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:44.697708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:44.700204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a46ff475-5ce3-411d-8c4c-fde825e0503a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:44.706300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 771/1519] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:44 2022] DELETE /identity/v3/projects/f4e069cf5bc84c01ba5171aec8179aa9 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:44.707711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55fb30-f722-4045-91fd-bceeec9460f8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:44.708000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55fb30-f722-4045-91fd-bceeec9460f8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:44.708276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55fb30-f722-4045-91fd-bceeec9460f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:44.735479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-dd55fb30-f722-4045-91fd-bceeec9460f8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:44.736722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-dd55fb30-f722-4045-91fd-bceeec9460f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:44.805965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:44.846908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 772/1520] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:44.855295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058f582-88a0-4daf-a4bf-44e3ad4d775e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:44.855295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058f582-88a0-4daf-a4bf-44e3ad4d775e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:44.855295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058f582-88a0-4daf-a4bf-44e3ad4d775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:44.859174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a6310201614d09939c820f49a5f2cb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:44.878412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a6310201614d09939c820f49a5f2cb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:44.887487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9058f582-88a0-4daf-a4bf-44e3ad4d775e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:44.888936 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9058f582-88a0-4daf-a4bf-44e3ad4d775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:44.902919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 773/1521] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:44.903837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058f582-88a0-4daf-a4bf-44e3ad4d775e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:44.922036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058f582-88a0-4daf-a4bf-44e3ad4d775e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:44.925347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:44.926123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:44.926878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:44.926878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:44.927415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:44.927776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:44.929443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:44.945811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 774/1522] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:38:44.953978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:44.968234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d679d-2eea-4b73-9df9-f0c5189c8a50 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:44.977147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:44.979325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:44.979325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:44.979325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:44.979325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:44.981391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:45.003721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a6310201614d09939c820f49a5f2cb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:45.004584 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a6310201614d09939c820f49a5f2cb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:45.019403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d33b511e-e7ac-4a28-bc7f-2f371a129e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a6310201614d09939c820f49a5f2cb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:45.030875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 749/1523] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:44 2022] DELETE /identity/v3/users/f5a6310201614d09939c820f49a5f2cb => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:45.035799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865a47-70af-474b-a4ef-a0dc3f031790 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.036437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865a47-70af-474b-a4ef-a0dc3f031790 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.037845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865a47-70af-474b-a4ef-a0dc3f031790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.065527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-73865a47-70af-474b-a4ef-a0dc3f031790 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:45.067053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-73865a47-70af-474b-a4ef-a0dc3f031790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:45.093675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 775/1524] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:44 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:45.095968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:45.100525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:45.197242 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa34e-1677-409b-9e40-f6139eb1be3f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:45.208947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:45.208947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.208947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.208947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.209277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:45.209277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:45.285136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 776/1525] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:45 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:38:45.286485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:45.294565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:45.306795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 750/1526] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:45.308091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73865a47-70af-474b-a4ef-a0dc3f031790 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:45.311376 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a0b52-37bb-489b-b568-b7445537c2af admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:45.324526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:45.324526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.324526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.324526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] PATH_INFO: `/v3/users/2930fa84635642ccb637e38019984ee4` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.324908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2930fa84635642ccb637e38019984ee4)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:45.324988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73865a47-70af-474b-a4ef-a0dc3f031790 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:45.325073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:45.328380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:45.329192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.329632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.330093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.331688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:45.334112 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:45.355297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 751/1527] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:45.355297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:45.355297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:45.414674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942e46e-df7d-4ec1-a3dc-b642be05fb13 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:45.418362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c8c48-64d0-47bf-96f8-ef783d60ba4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:45.418966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c8c48-64d0-47bf-96f8-ef783d60ba4d service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.419245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c8c48-64d0-47bf-96f8-ef783d60ba4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.419517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c8c48-64d0-47bf-96f8-ef783d60ba4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.428790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c8c48-64d0-47bf-96f8-ef783d60ba4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:45.430779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c8c48-64d0-47bf-96f8-ef783d60ba4d service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:45.440966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 752/1528] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:45.441885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c8c48-64d0-47bf-96f8-ef783d60ba4d service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:45.451313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c8c48-64d0-47bf-96f8-ef783d60ba4d service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:45.456297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe5b22-ea7c-4156-9cc1-a122b1581b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:45.456768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5b22-ea7c-4156-9cc1-a122b1581b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.457047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5b22-ea7c-4156-9cc1-a122b1581b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.457371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5b22-ea7c-4156-9cc1-a122b1581b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.458104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe5b22-ea7c-4156-9cc1-a122b1581b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:45.459890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe5b22-ea7c-4156-9cc1-a122b1581b70 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:45.475105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 753/1529] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:45.476320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe5b22-ea7c-4156-9cc1-a122b1581b70 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:45.489008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:45.489560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe5b22-ea7c-4156-9cc1-a122b1581b70 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:45.498095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842879fd-f88b-4250-84b3-84de4694ca38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:45.499154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842879fd-f88b-4250-84b3-84de4694ca38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.499432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842879fd-f88b-4250-84b3-84de4694ca38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.499752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842879fd-f88b-4250-84b3-84de4694ca38 admin admin] PATH_INFO: `/v3/projects/90633921fc274ec9b7f567af4ceb50af/users/3339f35875d94e60b6afff331ca67fbb/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.518702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842879fd-f88b-4250-84b3-84de4694ca38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90633921fc274ec9b7f567af4ceb50af, user_id=3339f35875d94e60b6afff331ca67fbb, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:45.521482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842879fd-f88b-4250-84b3-84de4694ca38 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:45.554994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2930fa84635642ccb637e38019984ee4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:45.584646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2930fa84635642ccb637e38019984ee4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:45.585774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2930fa84635642ccb637e38019984ee4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:45.606280 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 754/1530] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:45 2022] PUT /identity/v3/projects/90633921fc274ec9b7f567af4ceb50af/users/3339f35875d94e60b6afff331ca67fbb/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:45.616532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d57a2-8afd-446a-8bac-46ea2b293bd8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.617214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d57a2-8afd-446a-8bac-46ea2b293bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.617636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d57a2-8afd-446a-8bac-46ea2b293bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.658284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-aa7d57a2-8afd-446a-8bac-46ea2b293bd8 None None] MFA Rules not processed for user `3339f35875d94e60b6afff331ca67fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:45.660001 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-aa7d57a2-8afd-446a-8bac-46ea2b293bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:45.694503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2930fa84635642ccb637e38019984ee4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:45.708181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-84add176-70d5-470a-8e50-47d7989da59d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2930fa84635642ccb637e38019984ee4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:45.716073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 777/1531] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:45 2022] DELETE /identity/v3/users/2930fa84635642ccb637e38019984ee4 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:45.718698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487eb2a-4950-463c-802c-38d1b19f95f8 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.719228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487eb2a-4950-463c-802c-38d1b19f95f8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.719685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487eb2a-4950-463c-802c-38d1b19f95f8 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.725875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 778/1532] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:38:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:38:45.727679 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fec319-19b7-4b53-8ed8-129813d03015 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.732199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fec319-19b7-4b53-8ed8-129813d03015 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.732199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fec319-19b7-4b53-8ed8-129813d03015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.755067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a1fec319-19b7-4b53-8ed8-129813d03015 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:45.756713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a1fec319-19b7-4b53-8ed8-129813d03015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:45.767694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 755/1533] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:45.770630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b9b57-c2d7-45d3-bb02-30c3992a2f0e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.771195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b9b57-c2d7-45d3-bb02-30c3992a2f0e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.771546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b9b57-c2d7-45d3-bb02-30c3992a2f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.806726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-924b9b57-c2d7-45d3-bb02-30c3992a2f0e None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:45.809284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-924b9b57-c2d7-45d3-bb02-30c3992a2f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:45.875545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 779/1534] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:45.876427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fec319-19b7-4b53-8ed8-129813d03015 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:45.878581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a1fec319-19b7-4b53-8ed8-129813d03015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:45.891442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fec319-19b7-4b53-8ed8-129813d03015 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:45.897228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:45.897852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:45.898311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:45.898589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:45.899306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:45.901244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:45.966764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 756/1535] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:38:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:45.975040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b9b57-c2d7-45d3-bb02-30c3992a2f0e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:45.976550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-924b9b57-c2d7-45d3-bb02-30c3992a2f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:46.023674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 780/1536] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:45 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:46.024706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:46.031798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:46.071332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b9b57-c2d7-45d3-bb02-30c3992a2f0e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.153307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.154911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.155210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.155784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.188999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd456cd-0d9d-4517-b2d7-3aae2a3cce00 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.202985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4ad5-57ca-4ec5-a9a3-17519134db01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.202985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4ad5-57ca-4ec5-a9a3-17519134db01 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.202985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4ad5-57ca-4ec5-a9a3-17519134db01 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.202985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4ad5-57ca-4ec5-a9a3-17519134db01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.207472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e4ad5-57ca-4ec5-a9a3-17519134db01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.209240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e4ad5-57ca-4ec5-a9a3-17519134db01 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.222227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 781/1537] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:46.223202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4ad5-57ca-4ec5-a9a3-17519134db01 service nova] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:46.284818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:46.309075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.310792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.316251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4ad5-57ca-4ec5-a9a3-17519134db01 service nova] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.323070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581cabc-e3c9-4472-ac9f-e4e9fbe1938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.323692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581cabc-e3c9-4472-ac9f-e4e9fbe1938c admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.323972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581cabc-e3c9-4472-ac9f-e4e9fbe1938c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.324238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581cabc-e3c9-4472-ac9f-e4e9fbe1938c admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.324956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581cabc-e3c9-4472-ac9f-e4e9fbe1938c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.326918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581cabc-e3c9-4472-ac9f-e4e9fbe1938c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.334742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 757/1538] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:46.339502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:46.339502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:46.354805 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09fe9e-91ab-4f29-9752-455ecf9dd6d1 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.360878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.364326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.364326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.364326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] PATH_INFO: `/v3/projects/0ef7d8196e32442585ae469abb685fd6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.365312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef7d8196e32442585ae469abb685fd6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.367865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.458762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 782/1539] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:46 2022] POST /identity/v3/users => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:46.462717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581cabc-e3c9-4472-ac9f-e4e9fbe1938c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:46.473220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581cabc-e3c9-4472-ac9f-e4e9fbe1938c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.478266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8219fc-706a-42ed-86cf-53c627817380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.479100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8219fc-706a-42ed-86cf-53c627817380 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.479894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8219fc-706a-42ed-86cf-53c627817380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.480182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8219fc-706a-42ed-86cf-53c627817380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.481299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8219fc-706a-42ed-86cf-53c627817380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.484068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8219fc-706a-42ed-86cf-53c627817380 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.497614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 783/1540] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:46.502716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8219fc-706a-42ed-86cf-53c627817380 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:46.523456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8219fc-706a-42ed-86cf-53c627817380 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.547735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef7d8196e32442585ae469abb685fd6', 'request_id': 'req-78637462-dd80-4d4f-9548-4e888f5c03b4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:46.567747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef7d8196e32442585ae469abb685fd6', 'request_id': 'req-78637462-dd80-4d4f-9548-4e888f5c03b4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:46.581299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef7d8196e32442585ae469abb685fd6', 'request_id': 'req-78637462-dd80-4d4f-9548-4e888f5c03b4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:46.613967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.614734 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.614986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.615220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] PATH_INFO: `/v3/projects/d0b7426c18fd4199890871d0dc953b1f/users/41b16402a33642a599ec855fd2774edf/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.631315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b7426c18fd4199890871d0dc953b1f, user_id=41b16402a33642a599ec855fd2774edf, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.634106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.659764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] The token cache is being invalidate because project 0ef7d8196e32442585ae469abb685fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:46.660049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:46.660722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:46.662382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 758/1541] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:46 2022] DELETE /identity/v3/projects/0ef7d8196e32442585ae469abb685fd6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:46.666503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:46.668411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:46.684493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78637462-dd80-4d4f-9548-4e888f5c03b4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.704382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.704382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.704382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.704382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] PATH_INFO: `/v3/users/35f030aca17b4dc58548a25d5b3b89c7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.705070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f030aca17b4dc58548a25d5b3b89c7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.705070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.720157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 784/1542] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:46 2022] PUT /identity/v3/projects/d0b7426c18fd4199890871d0dc953b1f/users/41b16402a33642a599ec855fd2774edf/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:46.723989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:46.725877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:46.841005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afc982-5174-446f-9d9f-8c792a759fc0 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.857901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7a2e2-f541-44a7-910c-6fb0f0a3066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.858831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a2e2-f541-44a7-910c-6fb0f0a3066a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.860409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a2e2-f541-44a7-910c-6fb0f0a3066a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.869339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a2e2-f541-44a7-910c-6fb0f0a3066a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.869339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7a2e2-f541-44a7-910c-6fb0f0a3066a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.869339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7a2e2-f541-44a7-910c-6fb0f0a3066a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.919551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 785/1543] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:46.926165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7a2e2-f541-44a7-910c-6fb0f0a3066a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:46.958037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f030aca17b4dc58548a25d5b3b89c7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:46.961069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7a2e2-f541-44a7-910c-6fb0f0a3066a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:46.976062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652ef02-e444-4b90-8b24-87ae740d1b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:46.976752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652ef02-e444-4b90-8b24-87ae740d1b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:46.977029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652ef02-e444-4b90-8b24-87ae740d1b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:46.977298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652ef02-e444-4b90-8b24-87ae740d1b04 admin admin] PATH_INFO: `/v3/projects/d0b7426c18fd4199890871d0dc953b1f/users/41b16402a33642a599ec855fd2774edf/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:46.981565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652ef02-e444-4b90-8b24-87ae740d1b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b7426c18fd4199890871d0dc953b1f, user_id=41b16402a33642a599ec855fd2774edf, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:46.984269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652ef02-e444-4b90-8b24-87ae740d1b04 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:46.984633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f030aca17b4dc58548a25d5b3b89c7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:46.997013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 786/1544] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:46 2022] PUT /identity/v3/projects/d0b7426c18fd4199890871d0dc953b1f/users/41b16402a33642a599ec855fd2774edf/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:47.008816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a914d-f12c-45c4-8713-a515e029d8bb None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:47.009292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a914d-f12c-45c4-8713-a515e029d8bb None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:47.009643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a914d-f12c-45c4-8713-a515e029d8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:47.085653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-761a914d-f12c-45c4-8713-a515e029d8bb None None] MFA Rules not processed for user `41b16402a33642a599ec855fd2774edf`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:47.087048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-761a914d-f12c-45c4-8713-a515e029d8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:47.107268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f030aca17b4dc58548a25d5b3b89c7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:47.108130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f030aca17b4dc58548a25d5b3b89c7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:47.122707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f030aca17b4dc58548a25d5b3b89c7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:47.141069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 759/1545] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:46 2022] DELETE /identity/v3/users/35f030aca17b4dc58548a25d5b3b89c7 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:47.142063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:47.149013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:47.197600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 787/1546] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:47.236939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a914d-f12c-45c4-8713-a515e029d8bb None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:47.236939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-761a914d-f12c-45c4-8713-a515e029d8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:47.255137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:47.319061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaead65e-c307-4ed3-b643-c8282ed40ee5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:47.327998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:47.329644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:47.330112 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:47.331151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] PATH_INFO: `/v3/projects/b3a15af4aaf24498bdcfea58d8d23b20` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:47.331414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a914d-f12c-45c4-8713-a515e029d8bb None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:47.333296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3a15af4aaf24498bdcfea58d8d23b20)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:47.337333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:47.411256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:47.411990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:47.412273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:47.412724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:47.417237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:47.417386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:47.476749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3a15af4aaf24498bdcfea58d8d23b20', 'request_id': 'req-8a78e426-0f53-45a4-9352-baad1e1b85e7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:47.487718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3a15af4aaf24498bdcfea58d8d23b20', 'request_id': 'req-8a78e426-0f53-45a4-9352-baad1e1b85e7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:47.497731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3a15af4aaf24498bdcfea58d8d23b20', 'request_id': 'req-8a78e426-0f53-45a4-9352-baad1e1b85e7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:47.523208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:47.524787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:47.537496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 788/1547] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:47.646565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] The token cache is being invalidate because project b3a15af4aaf24498bdcfea58d8d23b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:47.646565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:47.646565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:47.647840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 760/1548] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:47 2022] DELETE /identity/v3/projects/b3a15af4aaf24498bdcfea58d8d23b20 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:47.651986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:47.677659 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609ffd1-cf32-4da9-9af1-0866d80335bd service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:47.758919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:47.760483 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:47.796811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:47.798538 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:47.799900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:47.800236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] PATH_INFO: `/v3/users/c58a2af6f3b1493a81baa37295dab6f4` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:47.803560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58a2af6f3b1493a81baa37295dab6f4)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:47.808503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:47.873300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78e426-0f53-45a4-9352-baad1e1b85e7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:47.878013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:47.878576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:47.878852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:47.879132 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] PATH_INFO: `/v3/projects/af344f72f4f443ff92df847143b494d5` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:47.880310 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af344f72f4f443ff92df847143b494d5)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:47.882223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:48.026757 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af344f72f4f443ff92df847143b494d5', 'request_id': 'req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.026918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58a2af6f3b1493a81baa37295dab6f4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.032970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af344f72f4f443ff92df847143b494d5', 'request_id': 'req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.039916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af344f72f4f443ff92df847143b494d5', 'request_id': 'req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.040064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58a2af6f3b1493a81baa37295dab6f4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.040835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58a2af6f3b1493a81baa37295dab6f4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.146570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] The token cache is being invalidate because project af344f72f4f443ff92df847143b494d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:48.146864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.153355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cff3be3a-4eb8-452a-ac5d-a3d959bd7ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.153355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 761/1549] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:47 2022] DELETE /identity/v3/projects/af344f72f4f443ff92df847143b494d5 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:48.156128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde83a52-709d-4879-8531-1e64038be1cf None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.156472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde83a52-709d-4879-8531-1e64038be1cf None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.156844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde83a52-709d-4879-8531-1e64038be1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.179433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bde83a52-709d-4879-8531-1e64038be1cf None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:48.180826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bde83a52-709d-4879-8531-1e64038be1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:48.223896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58a2af6f3b1493a81baa37295dab6f4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.232034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7648217b-2d03-471d-bb28-23ccb5006071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58a2af6f3b1493a81baa37295dab6f4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.238719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 789/1550] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:47 2022] DELETE /identity/v3/users/c58a2af6f3b1493a81baa37295dab6f4 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:48.241210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 762/1551] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:48.250464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b969a-e355-4a26-ab95-b97cca7e3135 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.250829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b969a-e355-4a26-ab95-b97cca7e3135 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.251174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b969a-e355-4a26-ab95-b97cca7e3135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.272170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-3b0b969a-e355-4a26-ab95-b97cca7e3135 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:48.274170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3b0b969a-e355-4a26-ab95-b97cca7e3135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:48.366630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 790/1552] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:48.373857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde83a52-709d-4879-8531-1e64038be1cf None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:48.383683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde83a52-709d-4879-8531-1e64038be1cf None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:48.390185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:48.391298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.391298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.391453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.392028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:48.392328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:48.394503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:48.410041 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 763/1553] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:38:48.421323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b969a-e355-4a26-ab95-b97cca7e3135 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:48.434687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b969a-e355-4a26-ab95-b97cca7e3135 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:48.440252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:48.441366 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.441714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.442123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.442864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:48.444965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:48.595357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 791/1554] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:48 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:48.599933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:48.612417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07ba4f-c973-41ec-9f1b-27a766d1cb42 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:48.705659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73137515-1f84-4fda-b14d-b5e761cbad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:48.706577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73137515-1f84-4fda-b14d-b5e761cbad14 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.706863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73137515-1f84-4fda-b14d-b5e761cbad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.707137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73137515-1f84-4fda-b14d-b5e761cbad14 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.707788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73137515-1f84-4fda-b14d-b5e761cbad14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:48.710016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73137515-1f84-4fda-b14d-b5e761cbad14 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:48.781696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:48.784572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:48.800980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7668bc-198f-4650-a464-3f1a9c2e3cd5 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:48.809856 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:48.810799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.812627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.813049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] PATH_INFO: `/v3/projects/d64ca8c11c1c4180b40d11d013a0f26c` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.818213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64ca8c11c1c4180b40d11d013a0f26c)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:48.820193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:48.835707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 764/1555] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:48 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:48.840012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73137515-1f84-4fda-b14d-b5e761cbad14 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:48.853745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73137515-1f84-4fda-b14d-b5e761cbad14 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:48.862212 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd5fcc-511a-4e87-95cd-6930bc5f0c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:48.862915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5fcc-511a-4e87-95cd-6930bc5f0c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.863256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5fcc-511a-4e87-95cd-6930bc5f0c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.863748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd5fcc-511a-4e87-95cd-6930bc5f0c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.864542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd5fcc-511a-4e87-95cd-6930bc5f0c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:48.866692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd5fcc-511a-4e87-95cd-6930bc5f0c13 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:48.882155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 765/1556] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:48.883218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd5fcc-511a-4e87-95cd-6930bc5f0c13 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:48.894792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd5fcc-511a-4e87-95cd-6930bc5f0c13 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:48.899393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df0393-4122-499d-90c7-cd9b0f13cb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:48.900018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df0393-4122-499d-90c7-cd9b0f13cb30 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.900296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df0393-4122-499d-90c7-cd9b0f13cb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.900620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df0393-4122-499d-90c7-cd9b0f13cb30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.901201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df0393-4122-499d-90c7-cd9b0f13cb30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:48.902822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df0393-4122-499d-90c7-cd9b0f13cb30 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:48.913583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 766/1557] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:48.917296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:48.917890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df0393-4122-499d-90c7-cd9b0f13cb30 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:48.927848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df0393-4122-499d-90c7-cd9b0f13cb30 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:48.932570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef56a7-39bc-4fab-bd06-30b842e7f7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:48.933220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef56a7-39bc-4fab-bd06-30b842e7f7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:48.933500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef56a7-39bc-4fab-bd06-30b842e7f7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:48.933772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef56a7-39bc-4fab-bd06-30b842e7f7e1 admin admin] PATH_INFO: `/v3/projects/6710ebddde994cf2b1efc375ab20e84b/users/e4a2d4f198c9478385d0d8860efe6dcf/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:48.956914 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef56a7-39bc-4fab-bd06-30b842e7f7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6710ebddde994cf2b1efc375ab20e84b, user_id=e4a2d4f198c9478385d0d8860efe6dcf, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:48.960082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef56a7-39bc-4fab-bd06-30b842e7f7e1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:48.965441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64ca8c11c1c4180b40d11d013a0f26c', 'request_id': 'req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.976406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64ca8c11c1c4180b40d11d013a0f26c', 'request_id': 'req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:48.983451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64ca8c11c1c4180b40d11d013a0f26c', 'request_id': 'req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:49.088740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 767/1558] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:48 2022] PUT /identity/v3/projects/6710ebddde994cf2b1efc375ab20e84b/users/e4a2d4f198c9478385d0d8860efe6dcf/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:49.100118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4680-2b43-4717-b055-4b5f9f323765 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.100516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4680-2b43-4717-b055-4b5f9f323765 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.100861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4680-2b43-4717-b055-4b5f9f323765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.130137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-0bef4680-2b43-4717-b055-4b5f9f323765 None None] MFA Rules not processed for user `e4a2d4f198c9478385d0d8860efe6dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:49.132816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:49.134123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0bef4680-2b43-4717-b055-4b5f9f323765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:49.138236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] The token cache is being invalidate because project d64ca8c11c1c4180b40d11d013a0f26c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:49.138572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:49.139625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3aad8c58-8cb1-4c17-b77f-c1e135d1c38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:49.141814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 792/1559] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:48 2022] DELETE /identity/v3/projects/d64ca8c11c1c4180b40d11d013a0f26c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:49.147282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1266ca-cc1e-4c0f-8139-82224ba3586b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.147570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1266ca-cc1e-4c0f-8139-82224ba3586b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.147848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1266ca-cc1e-4c0f-8139-82224ba3586b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.170671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-6b1266ca-cc1e-4c0f-8139-82224ba3586b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:49.172047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6b1266ca-cc1e-4c0f-8139-82224ba3586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:49.211211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 768/1560] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:49.215857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef4680-2b43-4717-b055-4b5f9f323765 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.217327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0bef4680-2b43-4717-b055-4b5f9f323765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:49.238403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 793/1561] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:49.246763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88767c-8519-4f20-894a-7421fbd540c9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.247051 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88767c-8519-4f20-894a-7421fbd540c9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.247326 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88767c-8519-4f20-894a-7421fbd540c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.267506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2e88767c-8519-4f20-894a-7421fbd540c9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:49.269263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2e88767c-8519-4f20-894a-7421fbd540c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:49.277133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 794/1562] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:49.281401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88767c-8519-4f20-894a-7421fbd540c9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.290348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88767c-8519-4f20-894a-7421fbd540c9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.291942 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef4680-2b43-4717-b055-4b5f9f323765 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.295008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.295586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.295867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.296140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.296455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0160e42-0082-4ddc-93d0-ab5052acab6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.296775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:49.297050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0160e42-0082-4ddc-93d0-ab5052acab6c admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.297144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.297326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0160e42-0082-4ddc-93d0-ab5052acab6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.297594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0160e42-0082-4ddc-93d0-ab5052acab6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.298247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0160e42-0082-4ddc-93d0-ab5052acab6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.298679 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.300139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0160e42-0082-4ddc-93d0-ab5052acab6c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.313298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 795/1563] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:38:49.314309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.324903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73bc9a-6078-4baf-a856-4bb9b1a3ab81 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.329760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816aa5f-a004-47a8-a7ea-16b094b9592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.331026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816aa5f-a004-47a8-a7ea-16b094b9592f admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.331309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816aa5f-a004-47a8-a7ea-16b094b9592f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.331581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816aa5f-a004-47a8-a7ea-16b094b9592f admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.332199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816aa5f-a004-47a8-a7ea-16b094b9592f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.334111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816aa5f-a004-47a8-a7ea-16b094b9592f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.384849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 769/1564] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:49 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:38:49.385979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0160e42-0082-4ddc-93d0-ab5052acab6c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.449522 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0160e42-0082-4ddc-93d0-ab5052acab6c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.454771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb9f40-2eba-4018-a9e7-a12f28f66c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.458900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb9f40-2eba-4018-a9e7-a12f28f66c82 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.458900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb9f40-2eba-4018-a9e7-a12f28f66c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.458900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb9f40-2eba-4018-a9e7-a12f28f66c82 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.458900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb9f40-2eba-4018-a9e7-a12f28f66c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.459381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb9f40-2eba-4018-a9e7-a12f28f66c82 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.467308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 796/1565] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:49 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:49.473894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816aa5f-a004-47a8-a7ea-16b094b9592f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.548329 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816aa5f-a004-47a8-a7ea-16b094b9592f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.557964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0c568-9175-449e-80fe-270e5c690916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.558686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0c568-9175-449e-80fe-270e5c690916 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.559067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0c568-9175-449e-80fe-270e5c690916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.559444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0c568-9175-449e-80fe-270e5c690916 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.560274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0c568-9175-449e-80fe-270e5c690916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.563089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0c568-9175-449e-80fe-270e5c690916 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.571540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 770/1566] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:49 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:49.576782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb9f40-2eba-4018-a9e7-a12f28f66c82 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.589793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb9f40-2eba-4018-a9e7-a12f28f66c82 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.596079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce494711-2bf9-4b23-8411-12cd301d013e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.596699 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494711-2bf9-4b23-8411-12cd301d013e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.600261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494711-2bf9-4b23-8411-12cd301d013e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.600261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494711-2bf9-4b23-8411-12cd301d013e admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.600261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce494711-2bf9-4b23-8411-12cd301d013e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.600261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce494711-2bf9-4b23-8411-12cd301d013e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.621526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 771/1567] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:49.622688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce494711-2bf9-4b23-8411-12cd301d013e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.638634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce494711-2bf9-4b23-8411-12cd301d013e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.645660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee2ec4-e7f3-4b1c-be38-8b03900c4afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.646851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee2ec4-e7f3-4b1c-be38-8b03900c4afd admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.647143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee2ec4-e7f3-4b1c-be38-8b03900c4afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.647419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee2ec4-e7f3-4b1c-be38-8b03900c4afd admin admin] PATH_INFO: `/v3/projects/553b51c96fae461ab898a2ecc0517e74/users/69690997ec6748318e5c9344c5b48bc8/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.670859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee2ec4-e7f3-4b1c-be38-8b03900c4afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=553b51c96fae461ab898a2ecc0517e74, user_id=69690997ec6748318e5c9344c5b48bc8, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.673493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 797/1568] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:49 2022] POST /identity/v3/users => generated 282 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:38:49.677263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee2ec4-e7f3-4b1c-be38-8b03900c4afd admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.680727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0c568-9175-449e-80fe-270e5c690916 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.694519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0c568-9175-449e-80fe-270e5c690916 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.701271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cdf23-572c-4b57-b99c-1b4d909440a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.702943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdf23-572c-4b57-b99c-1b4d909440a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.703337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdf23-572c-4b57-b99c-1b4d909440a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.703756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdf23-572c-4b57-b99c-1b4d909440a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.704519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cdf23-572c-4b57-b99c-1b4d909440a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.707186 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cdf23-572c-4b57-b99c-1b4d909440a0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.733036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 798/1569] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:49.736432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cdf23-572c-4b57-b99c-1b4d909440a0 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.747057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cdf23-572c-4b57-b99c-1b4d909440a0 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.752277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629f99b-4226-4059-a184-91720043e2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.755151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629f99b-4226-4059-a184-91720043e2fa admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.755577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629f99b-4226-4059-a184-91720043e2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.755943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629f99b-4226-4059-a184-91720043e2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.756563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629f99b-4226-4059-a184-91720043e2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.758872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629f99b-4226-4059-a184-91720043e2fa admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.780834 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 772/1570] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:49 2022] PUT /identity/v3/projects/553b51c96fae461ab898a2ecc0517e74/users/69690997ec6748318e5c9344c5b48bc8/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:49.786794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee2ec4-e7f3-4b1c-be38-8b03900c4afd admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.787844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 799/1571] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:49.789813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629f99b-4226-4059-a184-91720043e2fa admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.810681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629f99b-4226-4059-a184-91720043e2fa admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.874443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee2ec4-e7f3-4b1c-be38-8b03900c4afd admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.879519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6f575-5a52-4f55-8f86-f0c767c29218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.880195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6f575-5a52-4f55-8f86-f0c767c29218 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.880474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6f575-5a52-4f55-8f86-f0c767c29218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.880746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6f575-5a52-4f55-8f86-f0c767c29218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.881398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6f575-5a52-4f55-8f86-f0c767c29218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.883034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6f575-5a52-4f55-8f86-f0c767c29218 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.896402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 773/1572] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:49.899870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6f575-5a52-4f55-8f86-f0c767c29218 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:49.904434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1a049-6f9b-4076-b939-bcfef51da653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.905099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a049-6f9b-4076-b939-bcfef51da653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.905375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a049-6f9b-4076-b939-bcfef51da653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.905644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a049-6f9b-4076-b939-bcfef51da653 admin admin] PATH_INFO: `/v3/projects/ab16a60e410f4879b7b34b44217baaa6/users/34b3f7900b844581989376731d246407/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.911586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6f575-5a52-4f55-8f86-f0c767c29218 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:49.917474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dc4aa-cae0-4ae7-9aa9-9d365e3de067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:49.918423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dc4aa-cae0-4ae7-9aa9-9d365e3de067 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.918705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dc4aa-cae0-4ae7-9aa9-9d365e3de067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.918980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dc4aa-cae0-4ae7-9aa9-9d365e3de067 admin admin] PATH_INFO: `/v3/projects/553b51c96fae461ab898a2ecc0517e74/users/69690997ec6748318e5c9344c5b48bc8/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.925821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850dc4aa-cae0-4ae7-9aa9-9d365e3de067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=553b51c96fae461ab898a2ecc0517e74, user_id=69690997ec6748318e5c9344c5b48bc8, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.926437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1a049-6f9b-4076-b939-bcfef51da653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab16a60e410f4879b7b34b44217baaa6, user_id=34b3f7900b844581989376731d246407, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:49.926972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850dc4aa-cae0-4ae7-9aa9-9d365e3de067 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.929114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1a049-6f9b-4076-b939-bcfef51da653 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:49.937849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 774/1573] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:49 2022] PUT /identity/v3/projects/553b51c96fae461ab898a2ecc0517e74/users/69690997ec6748318e5c9344c5b48bc8/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:49.947801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f5bc-fd50-4ca0-8cd3-1e6e13d42a58 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:49.954256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f5bc-fd50-4ca0-8cd3-1e6e13d42a58 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:49.954256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2f5bc-fd50-4ca0-8cd3-1e6e13d42a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:49.977173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c2d2f5bc-fd50-4ca0-8cd3-1e6e13d42a58 None None] MFA Rules not processed for user `69690997ec6748318e5c9344c5b48bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:49.978496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c2d2f5bc-fd50-4ca0-8cd3-1e6e13d42a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:50.011582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 800/1574] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:49 2022] PUT /identity/v3/projects/ab16a60e410f4879b7b34b44217baaa6/users/34b3f7900b844581989376731d246407/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:50.023352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aea60-ee27-4f4a-a10b-90761339714d None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:50.023632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aea60-ee27-4f4a-a10b-90761339714d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:50.024041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aea60-ee27-4f4a-a10b-90761339714d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:50.055673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c73aea60-ee27-4f4a-a10b-90761339714d None None] MFA Rules not processed for user `34b3f7900b844581989376731d246407`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:50.057088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 775/1575] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:50.057214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c73aea60-ee27-4f4a-a10b-90761339714d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:50.069985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2f5bc-fd50-4ca0-8cd3-1e6e13d42a58 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:50.071642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c2d2f5bc-fd50-4ca0-8cd3-1e6e13d42a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:50.142656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2f5bc-fd50-4ca0-8cd3-1e6e13d42a58 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:50.150025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 801/1576] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:50.206557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46459be7-7aae-4f3f-9928-7461040e0d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:50.207279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46459be7-7aae-4f3f-9928-7461040e0d79 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:50.207630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46459be7-7aae-4f3f-9928-7461040e0d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:50.207969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46459be7-7aae-4f3f-9928-7461040e0d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:50.289907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46459be7-7aae-4f3f-9928-7461040e0d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:50.291620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46459be7-7aae-4f3f-9928-7461040e0d79 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:50.303301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 776/1577] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:50.307446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aea60-ee27-4f4a-a10b-90761339714d None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:50.307446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c73aea60-ee27-4f4a-a10b-90761339714d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:50.320432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aea60-ee27-4f4a-a10b-90761339714d None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:50.323825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a472666-2bfd-4df6-b546-5b9272011fa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:50.325400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472666-2bfd-4df6-b546-5b9272011fa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:50.325400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472666-2bfd-4df6-b546-5b9272011fa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:50.325400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472666-2bfd-4df6-b546-5b9272011fa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:50.359033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a472666-2bfd-4df6-b546-5b9272011fa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:50.360710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a472666-2bfd-4df6-b546-5b9272011fa2 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:50.397834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 802/1578] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:50.510781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46459be7-7aae-4f3f-9928-7461040e0d79 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:50.541513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:50.589121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46459be7-7aae-4f3f-9928-7461040e0d79 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:50.597624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:50.598374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:50.598829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:50.599008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] PATH_INFO: `/v3/users/e4a2d4f198c9478385d0d8860efe6dcf` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:50.600165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4a2d4f198c9478385d0d8860efe6dcf)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:50.608260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:50.803703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4a2d4f198c9478385d0d8860efe6dcf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:50.817047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4a2d4f198c9478385d0d8860efe6dcf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:50.894152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4a2d4f198c9478385d0d8860efe6dcf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:50.895075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4a2d4f198c9478385d0d8860efe6dcf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:50.902860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4a2d4f198c9478385d0d8860efe6dcf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:50.909513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 777/1579] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:50 2022] DELETE /identity/v3/users/e4a2d4f198c9478385d0d8860efe6dcf => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:50.913880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530cbeb-464b-4276-a575-5fdf45706643 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:50.914362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530cbeb-464b-4276-a575-5fdf45706643 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:50.914362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530cbeb-464b-4276-a575-5fdf45706643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:50.934176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a530cbeb-464b-4276-a575-5fdf45706643 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:50.935395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a530cbeb-464b-4276-a575-5fdf45706643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:51.001793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 803/1580] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:51.012602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:51.014625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:51.082243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865949cd-0343-4a2c-a88f-2e8930b3b2d0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:51.088625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b37a1-4f89-4b28-8cfb-a85ea400b4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:51.089494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b37a1-4f89-4b28-8cfb-a85ea400b4ca service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:51.089807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b37a1-4f89-4b28-8cfb-a85ea400b4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:51.090240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b37a1-4f89-4b28-8cfb-a85ea400b4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:51.114853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b37a1-4f89-4b28-8cfb-a85ea400b4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:51.116666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b37a1-4f89-4b28-8cfb-a85ea400b4ca service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:51.138373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 778/1581] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:51.414507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:51.818471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:51.878971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530cbeb-464b-4276-a575-5fdf45706643 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:51.880405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a530cbeb-464b-4276-a575-5fdf45706643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:51.900621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530cbeb-464b-4276-a575-5fdf45706643 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:51.906829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:51.907573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:51.907922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:51.908262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] PATH_INFO: `/v3/projects/6710ebddde994cf2b1efc375ab20e84b` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:51.910139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6710ebddde994cf2b1efc375ab20e84b)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:51.912104 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:52.047558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6710ebddde994cf2b1efc375ab20e84b', 'request_id': 'req-990b200e-9776-4cb6-943f-36e4a701c62d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.053368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6710ebddde994cf2b1efc375ab20e84b', 'request_id': 'req-990b200e-9776-4cb6-943f-36e4a701c62d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.060053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6710ebddde994cf2b1efc375ab20e84b', 'request_id': 'req-990b200e-9776-4cb6-943f-36e4a701c62d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.171669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] The token cache is being invalidate because project 6710ebddde994cf2b1efc375ab20e84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:52.172010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.172648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.174214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 804/1582] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:51 2022] DELETE /identity/v3/projects/6710ebddde994cf2b1efc375ab20e84b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:52.178383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b37a1-4f89-4b28-8cfb-a85ea400b4ca service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:52.188292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b37a1-4f89-4b28-8cfb-a85ea400b4ca service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:52.245768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:52.246428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:52.246706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:52.246979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] PATH_INFO: `/v3/users/69690997ec6748318e5c9344c5b48bc8` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:52.248090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69690997ec6748318e5c9344c5b48bc8)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:52.251001 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:52.498706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69690997ec6748318e5c9344c5b48bc8'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.524444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69690997ec6748318e5c9344c5b48bc8'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.592342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69690997ec6748318e5c9344c5b48bc8'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.593125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69690997ec6748318e5c9344c5b48bc8'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.603178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69690997ec6748318e5c9344c5b48bc8'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:52.613581 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 779/1583] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:52 2022] DELETE /identity/v3/users/69690997ec6748318e5c9344c5b48bc8 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:52.879302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:53.208665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:53.288752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:53.290469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:53.308814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b200e-9776-4cb6-943f-36e4a701c62d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:53.383332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:53.384058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:53.384338 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:53.384609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] PATH_INFO: `/v3/projects/553b51c96fae461ab898a2ecc0517e74` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:53.385740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=553b51c96fae461ab898a2ecc0517e74)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:53.387896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:53.500027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:53.502440 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:53.529171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553b51c96fae461ab898a2ecc0517e74', 'request_id': 'req-72f14cc7-795b-4a5a-8770-f46c0f8379e8'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:53.529171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553b51c96fae461ab898a2ecc0517e74', 'request_id': 'req-72f14cc7-795b-4a5a-8770-f46c0f8379e8'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:53.535537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553b51c96fae461ab898a2ecc0517e74', 'request_id': 'req-72f14cc7-795b-4a5a-8770-f46c0f8379e8'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:53.598498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cec3-45e7-4375-bb64-2b98dda00b31 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:53.624066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] The token cache is being invalidate because project 553b51c96fae461ab898a2ecc0517e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:53.626719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:53.626719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72f14cc7-795b-4a5a-8770-f46c0f8379e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:53.631037 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 805/1584] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:53 2022] DELETE /identity/v3/projects/553b51c96fae461ab898a2ecc0517e74 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:53.632183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508d162-4f3a-4b31-a9d8-79da69755fc3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:53.632516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508d162-4f3a-4b31-a9d8-79da69755fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:53.632791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508d162-4f3a-4b31-a9d8-79da69755fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:53.654456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-f508d162-4f3a-4b31-a9d8-79da69755fc3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:53.655707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f508d162-4f3a-4b31-a9d8-79da69755fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:53.663987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:53.664718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:53.665058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:53.665527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:53.667454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:53.745857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 806/1585] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:53.754958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012020-1249-491d-bca0-98bb80df9c70 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:53.755248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012020-1249-491d-bca0-98bb80df9c70 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:53.755511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:53.755620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012020-1249-491d-bca0-98bb80df9c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:53.756928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:53.766873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 780/1586] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:53.781377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c2012020-1249-491d-bca0-98bb80df9c70 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:53.782818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c2012020-1249-491d-bca0-98bb80df9c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:53.792889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 807/1587] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:53.796921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:53.808586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300c369-3095-4ade-95c6-1dfc4b607567 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:53.813687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:53.814109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:53.814453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:53.814821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:53.815653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:53.816025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:53.817727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:53.834124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 781/1588] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:38:53.838495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2012020-1249-491d-bca0-98bb80df9c70 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:53.851921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2012020-1249-491d-bca0-98bb80df9c70 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:53.858412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830435a-4b4f-41f2-9a60-0ff01799d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:53.859481 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830435a-4b4f-41f2-9a60-0ff01799d367 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:53.859978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830435a-4b4f-41f2-9a60-0ff01799d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:53.860463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830435a-4b4f-41f2-9a60-0ff01799d367 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:53.861415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830435a-4b4f-41f2-9a60-0ff01799d367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:53.866275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830435a-4b4f-41f2-9a60-0ff01799d367 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:53.959102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 808/1589] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:53 2022] POST /identity/v3/projects => generated 409 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:53.963377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:53.974198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e5d9b-ff38-4e13-9c72-d53dd9d64cb5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:54.033025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed53bb-102b-41b2-a025-26e76926a251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:54.034252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed53bb-102b-41b2-a025-26e76926a251 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:54.034537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed53bb-102b-41b2-a025-26e76926a251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:54.034812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed53bb-102b-41b2-a025-26e76926a251 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:54.035454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed53bb-102b-41b2-a025-26e76926a251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:54.037377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed53bb-102b-41b2-a025-26e76926a251 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:54.068098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:54.139276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 782/1590] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:53 2022] POST /identity/v3/users => generated 305 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:38:54.152235 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830435a-4b4f-41f2-9a60-0ff01799d367 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:54.180143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830435a-4b4f-41f2-9a60-0ff01799d367 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:54.194606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8701c-39dd-4941-89d9-8dbd791466d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:54.195308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8701c-39dd-4941-89d9-8dbd791466d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:54.196806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8701c-39dd-4941-89d9-8dbd791466d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:54.197263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8701c-39dd-4941-89d9-8dbd791466d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:54.198147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8701c-39dd-4941-89d9-8dbd791466d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:54.203048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8701c-39dd-4941-89d9-8dbd791466d5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:54.227144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 809/1591] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:54.234201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed53bb-102b-41b2-a025-26e76926a251 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:54.259465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed53bb-102b-41b2-a025-26e76926a251 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:54.265011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29152a49-9749-402b-b435-2a744c88be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:54.265675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152a49-9749-402b-b435-2a744c88be3b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:54.266524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152a49-9749-402b-b435-2a744c88be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:54.266524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152a49-9749-402b-b435-2a744c88be3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:54.266900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29152a49-9749-402b-b435-2a744c88be3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:54.269049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29152a49-9749-402b-b435-2a744c88be3b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:54.281352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 783/1592] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:54.286520 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8701c-39dd-4941-89d9-8dbd791466d5 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:54.303445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8701c-39dd-4941-89d9-8dbd791466d5 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:54.309925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75bb2-3e34-437c-8632-0231f6fcac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:54.310642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75bb2-3e34-437c-8632-0231f6fcac1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:54.310929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75bb2-3e34-437c-8632-0231f6fcac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:54.311247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e75bb2-3e34-437c-8632-0231f6fcac1a admin admin] PATH_INFO: `/v3/projects/71a87986b04c430d970420b45231061f/users/6ec8b89213554c65a6d534428094d569/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:54.331348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e75bb2-3e34-437c-8632-0231f6fcac1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a87986b04c430d970420b45231061f, user_id=6ec8b89213554c65a6d534428094d569, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:54.334802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e75bb2-3e34-437c-8632-0231f6fcac1a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:54.420987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:54.439038 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 810/1593] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:54 2022] PUT /identity/v3/projects/71a87986b04c430d970420b45231061f/users/6ec8b89213554c65a6d534428094d569/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:54.449386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f1ed7-9324-42c6-8e6d-50f19a96c52b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:54.449671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f1ed7-9324-42c6-8e6d-50f19a96c52b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:54.450030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f1ed7-9324-42c6-8e6d-50f19a96c52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:54.480873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f93f1ed7-9324-42c6-8e6d-50f19a96c52b None None] MFA Rules not processed for user `6ec8b89213554c65a6d534428094d569`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:54.483366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f93f1ed7-9324-42c6-8e6d-50f19a96c52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:54.595638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 784/1594] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:54.597268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75bb2-3e34-437c-8632-0231f6fcac1a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:54.750440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e75bb2-3e34-437c-8632-0231f6fcac1a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:54.756586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f55086-dc03-43cb-9ba9-bc414883beb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:54.757344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f55086-dc03-43cb-9ba9-bc414883beb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:54.757727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f55086-dc03-43cb-9ba9-bc414883beb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:54.758134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f55086-dc03-43cb-9ba9-bc414883beb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:54.758941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f55086-dc03-43cb-9ba9-bc414883beb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:54.761118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f55086-dc03-43cb-9ba9-bc414883beb9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:54.876961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 811/1595] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:54 2022] POST /identity/v3/projects => generated 407 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:54.880149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f1ed7-9324-42c6-8e6d-50f19a96c52b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:54.891654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f1ed7-9324-42c6-8e6d-50f19a96c52b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:54.965508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:54.966170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:54.966451 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:54.966726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:54.967418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:54.969391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:55.123713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:55.132738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 785/1596] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:54 2022] POST /identity/v3/users => generated 310 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:55.136689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f55086-dc03-43cb-9ba9-bc414883beb9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:55.149701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f55086-dc03-43cb-9ba9-bc414883beb9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:55.151174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:55.152963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:55.155254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465d7f7-3fec-4f2c-8e85-a3c8741d650d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:55.155945 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465d7f7-3fec-4f2c-8e85-a3c8741d650d admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.156419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465d7f7-3fec-4f2c-8e85-a3c8741d650d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.156879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465d7f7-3fec-4f2c-8e85-a3c8741d650d admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.157743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465d7f7-3fec-4f2c-8e85-a3c8741d650d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:55.159460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465d7f7-3fec-4f2c-8e85-a3c8741d650d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:55.170895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7db8f6-4447-4067-8531-66fae0af9f03 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:55.174688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 812/1597] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:55.178163 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:55.178989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.181649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.182325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465d7f7-3fec-4f2c-8e85-a3c8741d650d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:55.182628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] PATH_INFO: `/v3/users/09622c9333e749f99a5f9fdfefc3f91f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.184634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09622c9333e749f99a5f9fdfefc3f91f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:55.187309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:55.194711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465d7f7-3fec-4f2c-8e85-a3c8741d650d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:55.206367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b257f2-52cf-48b4-9e4f-61c8468f2add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:55.207671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b257f2-52cf-48b4-9e4f-61c8468f2add admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.207671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b257f2-52cf-48b4-9e4f-61c8468f2add admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.208083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b257f2-52cf-48b4-9e4f-61c8468f2add admin admin] PATH_INFO: `/v3/projects/c68b8d48007640e58ed94846648b3eec/users/327ff41d31754fd8930e7e8892a96a2e/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.227184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b257f2-52cf-48b4-9e4f-61c8468f2add admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68b8d48007640e58ed94846648b3eec, user_id=327ff41d31754fd8930e7e8892a96a2e, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:55.229897 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b257f2-52cf-48b4-9e4f-61c8468f2add admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:55.363704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 813/1598] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:55 2022] PUT /identity/v3/projects/c68b8d48007640e58ed94846648b3eec/users/327ff41d31754fd8930e7e8892a96a2e/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:55.367290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b257f2-52cf-48b4-9e4f-61c8468f2add admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:55.452880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b257f2-52cf-48b4-9e4f-61c8468f2add admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:55.458632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73374ffd-5dd2-471f-8f69-ce8901cf726b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:55.459618 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374ffd-5dd2-471f-8f69-ce8901cf726b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.460047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374ffd-5dd2-471f-8f69-ce8901cf726b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.460453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374ffd-5dd2-471f-8f69-ce8901cf726b admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.461273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73374ffd-5dd2-471f-8f69-ce8901cf726b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:55.465329 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73374ffd-5dd2-471f-8f69-ce8901cf726b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:55.482445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 814/1599] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:38:55.486164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73374ffd-5dd2-471f-8f69-ce8901cf726b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:55.497331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09622c9333e749f99a5f9fdfefc3f91f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:55.498009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73374ffd-5dd2-471f-8f69-ce8901cf726b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:55.504407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfb0ab-8f38-43b3-be62-7a9b06e257db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:55.505657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfb0ab-8f38-43b3-be62-7a9b06e257db admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.506155 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfb0ab-8f38-43b3-be62-7a9b06e257db admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.506580 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfb0ab-8f38-43b3-be62-7a9b06e257db admin admin] PATH_INFO: `/v3/projects/c68b8d48007640e58ed94846648b3eec/users/327ff41d31754fd8930e7e8892a96a2e/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.509524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfb0ab-8f38-43b3-be62-7a9b06e257db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68b8d48007640e58ed94846648b3eec, user_id=327ff41d31754fd8930e7e8892a96a2e, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:55.512408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfb0ab-8f38-43b3-be62-7a9b06e257db admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:55.528583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09622c9333e749f99a5f9fdfefc3f91f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:55.534688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 815/1600] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:55 2022] PUT /identity/v3/projects/c68b8d48007640e58ed94846648b3eec/users/327ff41d31754fd8930e7e8892a96a2e/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:55.546057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da806a8-5b6f-4c2d-9301-306d8a5af699 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.548127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da806a8-5b6f-4c2d-9301-306d8a5af699 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.548634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da806a8-5b6f-4c2d-9301-306d8a5af699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.595458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7da806a8-5b6f-4c2d-9301-306d8a5af699 None None] MFA Rules not processed for user `327ff41d31754fd8930e7e8892a96a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:55.597739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7da806a8-5b6f-4c2d-9301-306d8a5af699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:55.631044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:55.631990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09622c9333e749f99a5f9fdfefc3f91f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:55.633029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09622c9333e749f99a5f9fdfefc3f91f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:55.645480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-10e702d3-ef89-42f4-9d43-fa5ea9512124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09622c9333e749f99a5f9fdfefc3f91f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:55.660039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 786/1601] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:55 2022] DELETE /identity/v3/users/09622c9333e749f99a5f9fdfefc3f91f => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:55.675607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363523a-c745-4f67-8a1b-3bc74d1494bb None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.675607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363523a-c745-4f67-8a1b-3bc74d1494bb None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.675607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363523a-c745-4f67-8a1b-3bc74d1494bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.696764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4363523a-c745-4f67-8a1b-3bc74d1494bb None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:55.698928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4363523a-c745-4f67-8a1b-3bc74d1494bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:55.715786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 816/1602] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:55.735751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da806a8-5b6f-4c2d-9301-306d8a5af699 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:55.740722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7da806a8-5b6f-4c2d-9301-306d8a5af699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:55.803877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 787/1603] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:55.819216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363523a-c745-4f67-8a1b-3bc74d1494bb None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:55.821110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4363523a-c745-4f67-8a1b-3bc74d1494bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:55.875323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da806a8-5b6f-4c2d-9301-306d8a5af699 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:55.938106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363523a-c745-4f67-8a1b-3bc74d1494bb None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:55.943021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb46bd-f43d-444b-9398-fd1bd099cfe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:55.944206 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb46bd-f43d-444b-9398-fd1bd099cfe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.944579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb46bd-f43d-444b-9398-fd1bd099cfe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.944998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb46bd-f43d-444b-9398-fd1bd099cfe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.956184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:55.956808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:55.957088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:55.957360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:55.963996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb46bd-f43d-444b-9398-fd1bd099cfe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:55.965467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb46bd-f43d-444b-9398-fd1bd099cfe8 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:55.988992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 788/1604] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:56.043207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:56.045488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:56.055059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:56.064434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 817/1605] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:55 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:38:56.619807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb46bd-f43d-444b-9398-fd1bd099cfe8 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:56.631420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:56.632908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:56.634363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb46bd-f43d-444b-9398-fd1bd099cfe8 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:56.639219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:56.639989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:56.640563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:56.640838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:56.642629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:56.649788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2a90d-f08f-4a20-8ef0-32e767363c42 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:56.652253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:56.657264 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:56.658083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:56.658414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:56.659600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] PATH_INFO: `/v3/projects/4856d9f3058c49b2ae56c3e95d878684` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:56.660953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4856d9f3058c49b2ae56c3e95d878684)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:56.663467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:56.750301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:56.753565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:56.774071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 789/1606] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:56 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:56.807961 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4856d9f3058c49b2ae56c3e95d878684', 'request_id': 'req-b1e534a1-b9dd-42c4-827a-fb10463ee78d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:56.816615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4856d9f3058c49b2ae56c3e95d878684', 'request_id': 'req-b1e534a1-b9dd-42c4-827a-fb10463ee78d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:56.825127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4856d9f3058c49b2ae56c3e95d878684', 'request_id': 'req-b1e534a1-b9dd-42c4-827a-fb10463ee78d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:56.899866 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] The token cache is being invalidate because project 4856d9f3058c49b2ae56c3e95d878684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:56.900163 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:56.902290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:56.903105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 818/1607] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:56 2022] DELETE /identity/v3/projects/4856d9f3058c49b2ae56c3e95d878684 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:56.907161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:56.928861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b28f9-4967-4ca9-993c-2ec520175e6f service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:57.000988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:57.001866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:57.002194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:57.002758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] PATH_INFO: `/v3/users/5ae17868764249fd8eb656dc8ddcb08a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:57.004130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae17868764249fd8eb656dc8ddcb08a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:57.011615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:57.325795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae17868764249fd8eb656dc8ddcb08a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:57.344521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae17868764249fd8eb656dc8ddcb08a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:57.409007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae17868764249fd8eb656dc8ddcb08a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:57.409690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae17868764249fd8eb656dc8ddcb08a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:57.418998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0557734a-8ebb-41b2-b0da-03c8d5999b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae17868764249fd8eb656dc8ddcb08a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:57.425898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 790/1608] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:38:56 2022] DELETE /identity/v3/users/5ae17868764249fd8eb656dc8ddcb08a => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:57.739975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:57.925716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:57.938052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:57.980800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e534a1-b9dd-42c4-827a-fb10463ee78d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:58.053048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:58.053968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:58.054469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:58.054894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] PATH_INFO: `/v3/projects/eaefb32081c64da88c529c889a958419` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:58.056544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaefb32081c64da88c529c889a958419)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:58.058914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:58.212994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaefb32081c64da88c529c889a958419', 'request_id': 'req-bb904939-e793-4d5b-95e1-0c9f4bfb1521'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:58.220876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaefb32081c64da88c529c889a958419', 'request_id': 'req-bb904939-e793-4d5b-95e1-0c9f4bfb1521'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:58.235628 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaefb32081c64da88c529c889a958419', 'request_id': 'req-bb904939-e793-4d5b-95e1-0c9f4bfb1521'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:58.369737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] The token cache is being invalidate because project eaefb32081c64da88c529c889a958419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:38:58.370059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:58.371214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bb904939-e793-4d5b-95e1-0c9f4bfb1521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:38:58.377740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 819/1609] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:38:57 2022] DELETE /identity/v3/projects/eaefb32081c64da88c529c889a958419 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:58.379692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afa10c-c71a-4cc8-9a56-1dec6a504a18 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:58.380547 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afa10c-c71a-4cc8-9a56-1dec6a504a18 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:58.380994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afa10c-c71a-4cc8-9a56-1dec6a504a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:58.403743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-02afa10c-c71a-4cc8-9a56-1dec6a504a18 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:58.405026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-02afa10c-c71a-4cc8-9a56-1dec6a504a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:58.477156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 791/1610] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:38:58.486470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42f7d9-5df1-4d5e-9aef-15bab40be3e9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:58.487357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42f7d9-5df1-4d5e-9aef-15bab40be3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:58.487635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42f7d9-5df1-4d5e-9aef-15bab40be3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:58.509993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5c42f7d9-5df1-4d5e-9aef-15bab40be3e9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:58.511336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5c42f7d9-5df1-4d5e-9aef-15bab40be3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:58.519811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 820/1611] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:58.524306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afa10c-c71a-4cc8-9a56-1dec6a504a18 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:58.535960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afa10c-c71a-4cc8-9a56-1dec6a504a18 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:58.541653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:58.542309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:58.542588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:58.542857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:58.543509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:38:58.543933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:58.545836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:58.563670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 792/1612] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:38:58.564832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42f7d9-5df1-4d5e-9aef-15bab40be3e9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:58.575106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42f7d9-5df1-4d5e-9aef-15bab40be3e9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:58.582168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ad842-8987-4b48-b002-30730e858f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:58.582710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ad842-8987-4b48-b002-30730e858f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:58.583112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ad842-8987-4b48-b002-30730e858f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:58.583464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ad842-8987-4b48-b002-30730e858f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:58.584181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ad842-8987-4b48-b002-30730e858f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:58.586128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ad842-8987-4b48-b002-30730e858f1c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:58.722546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 821/1613] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:58 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:58.723952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:58.742792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49a253-da2f-41a5-ad0d-232528bd21db admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:58.806543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eed5d1-7dde-433d-b80c-e12e1b174ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:58.807042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed5d1-7dde-433d-b80c-e12e1b174ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:58.807326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed5d1-7dde-433d-b80c-e12e1b174ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:58.807611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eed5d1-7dde-433d-b80c-e12e1b174ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:58.808247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eed5d1-7dde-433d-b80c-e12e1b174ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:58.810175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eed5d1-7dde-433d-b80c-e12e1b174ee1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:58.958622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 793/1614] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:58 2022] POST /identity/v3/users => generated 284 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:58.962278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ad842-8987-4b48-b002-30730e858f1c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:58.974734 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ad842-8987-4b48-b002-30730e858f1c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:58.980357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb168e-6684-458b-b82e-534fe169f796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:58.981223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb168e-6684-458b-b82e-534fe169f796 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:58.981740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb168e-6684-458b-b82e-534fe169f796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:58.982140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb168e-6684-458b-b82e-534fe169f796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:58.982984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb168e-6684-458b-b82e-534fe169f796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:58.984818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb168e-6684-458b-b82e-534fe169f796 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:58.999158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 822/1615] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:59.001319 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eed5d1-7dde-433d-b80c-e12e1b174ee1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:59.011568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eed5d1-7dde-433d-b80c-e12e1b174ee1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:59.016860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67b920-c696-42d6-8609-422fb999e831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:59.017469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67b920-c696-42d6-8609-422fb999e831 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:59.017749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67b920-c696-42d6-8609-422fb999e831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:59.018912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67b920-c696-42d6-8609-422fb999e831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:59.024567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67b920-c696-42d6-8609-422fb999e831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:59.024567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67b920-c696-42d6-8609-422fb999e831 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:59.035764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 794/1616] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:59.037096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb168e-6684-458b-b82e-534fe169f796 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:59.047596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb168e-6684-458b-b82e-534fe169f796 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:59.052555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59994295-c84a-4a44-be34-7d9b982903f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:59.053272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994295-c84a-4a44-be34-7d9b982903f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:59.053551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994295-c84a-4a44-be34-7d9b982903f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:59.053819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59994295-c84a-4a44-be34-7d9b982903f5 admin admin] PATH_INFO: `/v3/projects/6588debbf9374d8f97af3ce21f0be38d/users/f6738fcb683c4230954259fd6615750a/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:59.069624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59994295-c84a-4a44-be34-7d9b982903f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6588debbf9374d8f97af3ce21f0be38d, user_id=f6738fcb683c4230954259fd6615750a, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:59.072393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59994295-c84a-4a44-be34-7d9b982903f5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:59.092624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:59.195540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 823/1617] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:59 2022] PUT /identity/v3/projects/6588debbf9374d8f97af3ce21f0be38d/users/f6738fcb683c4230954259fd6615750a/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:38:59.206261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c95560-258c-4885-99e3-e132a4224d2d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:59.206543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c95560-258c-4885-99e3-e132a4224d2d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:59.206813 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c95560-258c-4885-99e3-e132a4224d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:59.235048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e5c95560-258c-4885-99e3-e132a4224d2d None None] MFA Rules not processed for user `f6738fcb683c4230954259fd6615750a`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:59.236313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e5c95560-258c-4885-99e3-e132a4224d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:59.305540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 795/1618] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:38:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:59.310354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59994295-c84a-4a44-be34-7d9b982903f5 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:59.374710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59994295-c84a-4a44-be34-7d9b982903f5 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:59.380006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf9c3e-8911-4d2a-b228-10c87a7ded33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:59.380618 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf9c3e-8911-4d2a-b228-10c87a7ded33 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:59.380898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf9c3e-8911-4d2a-b228-10c87a7ded33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:59.381172 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf9c3e-8911-4d2a-b228-10c87a7ded33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:59.381816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf9c3e-8911-4d2a-b228-10c87a7ded33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:59.383765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf9c3e-8911-4d2a-b228-10c87a7ded33 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:59.539120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 824/1619] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:38:59 2022] POST /identity/v3/projects => generated 367 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:59.540724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c95560-258c-4885-99e3-e132a4224d2d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:59.561413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c95560-258c-4885-99e3-e132a4224d2d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:59.618307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e55756-13db-4b65-a48e-09212fc41085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:59.618839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e55756-13db-4b65-a48e-09212fc41085 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:59.619251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e55756-13db-4b65-a48e-09212fc41085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:59.619613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e55756-13db-4b65-a48e-09212fc41085 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:59.620275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e55756-13db-4b65-a48e-09212fc41085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:59.622220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e55756-13db-4b65-a48e-09212fc41085 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:59.723031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 796/1620] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:38:59 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:38:59.735570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf9c3e-8911-4d2a-b228-10c87a7ded33 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:59.746758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf9c3e-8911-4d2a-b228-10c87a7ded33 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:59.751587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc6665-cef3-42de-9d03-165ea1fe6e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:59.752893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6665-cef3-42de-9d03-165ea1fe6e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:59.753192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6665-cef3-42de-9d03-165ea1fe6e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:59.753541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6665-cef3-42de-9d03-165ea1fe6e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:59.754295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc6665-cef3-42de-9d03-165ea1fe6e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:59.756209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc6665-cef3-42de-9d03-165ea1fe6e67 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:59.767759 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 825/1621] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:38:59.772093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e55756-13db-4b65-a48e-09212fc41085 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:59.784447 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e55756-13db-4b65-a48e-09212fc41085 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:59.789711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d4031-09da-40f1-91cd-393559dcd95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:59.790523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4031-09da-40f1-91cd-393559dcd95d admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:59.790807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4031-09da-40f1-91cd-393559dcd95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:59.791117 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4031-09da-40f1-91cd-393559dcd95d admin admin] PATH_INFO: `/v3/projects/121d1876ac644b5ca30b68508274ea7f/users/966f3a1abf7f4a60933b4c1cb529d26e/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:59.807475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d4031-09da-40f1-91cd-393559dcd95d admin admin] RBAC: Authorizing `identity:create_grant(project_id=121d1876ac644b5ca30b68508274ea7f, user_id=966f3a1abf7f4a60933b4c1cb529d26e, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:59.809710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:59.810620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d4031-09da-40f1-91cd-393559dcd95d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:59.912303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 797/1622] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:38:59 2022] PUT /identity/v3/projects/121d1876ac644b5ca30b68508274ea7f/users/966f3a1abf7f4a60933b4c1cb529d26e/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:38:59.916371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc6665-cef3-42de-9d03-165ea1fe6e67 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:59.926978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc6665-cef3-42de-9d03-165ea1fe6e67 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:59.983137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62035d26-7e20-47b0-94f0-59489edf799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:59.983779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035d26-7e20-47b0-94f0-59489edf799f admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:59.984014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035d26-7e20-47b0-94f0-59489edf799f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:59.984340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62035d26-7e20-47b0-94f0-59489edf799f admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:59.984977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62035d26-7e20-47b0-94f0-59489edf799f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:59.987234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62035d26-7e20-47b0-94f0-59489edf799f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:59.997091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 826/1623] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:00.001532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d4031-09da-40f1-91cd-393559dcd95d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:00.011880 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d4031-09da-40f1-91cd-393559dcd95d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:00.016468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:00.017109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:00.017419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:00.017692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] PATH_INFO: `/v3/projects/121d1876ac644b5ca30b68508274ea7f/users/966f3a1abf7f4a60933b4c1cb529d26e/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:00.019833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121d1876ac644b5ca30b68508274ea7f, user_id=966f3a1abf7f4a60933b4c1cb529d26e, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:00.022491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:00.032676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 798/1624] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:00 2022] PUT /identity/v3/projects/121d1876ac644b5ca30b68508274ea7f/users/966f3a1abf7f4a60933b4c1cb529d26e/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:00.048023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad55fc-67aa-409f-9bd5-7af6e17bac03 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:00.048023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad55fc-67aa-409f-9bd5-7af6e17bac03 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:00.048023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad55fc-67aa-409f-9bd5-7af6e17bac03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:00.077818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5bad55fc-67aa-409f-9bd5-7af6e17bac03 None None] MFA Rules not processed for user `966f3a1abf7f4a60933b4c1cb529d26e`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:00.079110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5bad55fc-67aa-409f-9bd5-7af6e17bac03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:00.110253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:00.158292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 827/1625] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:00.170422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:00.172118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:00.230632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b9080-e297-4010-ac7f-5899b8533805 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:00.234248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:00.234856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:00.235167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:00.235437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:00.244750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:00.246269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:00.255799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 799/1626] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:00.385260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad55fc-67aa-409f-9bd5-7af6e17bac03 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:00.396133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad55fc-67aa-409f-9bd5-7af6e17bac03 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:00.399850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80365393-bb01-468b-bfbb-8c81e373da98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:00.400463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80365393-bb01-468b-bfbb-8c81e373da98 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:00.400742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80365393-bb01-468b-bfbb-8c81e373da98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:00.401044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80365393-bb01-468b-bfbb-8c81e373da98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:00.442834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:00.460815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80365393-bb01-468b-bfbb-8c81e373da98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:00.462328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80365393-bb01-468b-bfbb-8c81e373da98 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:00.472572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 828/1627] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:00.591028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:00.592604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:00.637486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:00.656993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756116f-bcc4-46b1-9b29-ab1daa39f334 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:00.661686 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:00.662349 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:00.662665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:00.662937 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] PATH_INFO: `/v3/users/3339f35875d94e60b6afff331ca67fbb` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:00.664048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3339f35875d94e60b6afff331ca67fbb)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:00.665965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:00.772977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:00.891589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3339f35875d94e60b6afff331ca67fbb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:00.904901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3339f35875d94e60b6afff331ca67fbb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.006766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3339f35875d94e60b6afff331ca67fbb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.007389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3339f35875d94e60b6afff331ca67fbb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.015379 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3339f35875d94e60b6afff331ca67fbb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.021560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 800/1628] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:00 2022] DELETE /identity/v3/users/3339f35875d94e60b6afff331ca67fbb => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:01.026196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612afa6-dbd5-45a4-8d88-3eaaf1417b6e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:01.026663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612afa6-dbd5-45a4-8d88-3eaaf1417b6e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:01.026753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612afa6-dbd5-45a4-8d88-3eaaf1417b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:01.045631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-8612afa6-dbd5-45a4-8d88-3eaaf1417b6e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:01.046852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8612afa6-dbd5-45a4-8d88-3eaaf1417b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:01.106368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 829/1629] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:01.117252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:01.118789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:01.182187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0037e70-0c0f-4b9c-bbca-413d1b26f253 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:01.185881 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6974d73-c9eb-4001-b71a-3b9a61a64da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:01.186516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974d73-c9eb-4001-b71a-3b9a61a64da0 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:01.186796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974d73-c9eb-4001-b71a-3b9a61a64da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:01.187067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6974d73-c9eb-4001-b71a-3b9a61a64da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:01.192145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:01.196808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6974d73-c9eb-4001-b71a-3b9a61a64da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:01.198272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6974d73-c9eb-4001-b71a-3b9a61a64da0 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:01.207588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 801/1630] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:01.237901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:01.688349 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612afa6-dbd5-45a4-8d88-3eaaf1417b6e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:01.689835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8612afa6-dbd5-45a4-8d88-3eaaf1417b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:01.702096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612afa6-dbd5-45a4-8d88-3eaaf1417b6e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:01.709177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:01.714584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:01.714584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:01.714584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] PATH_INFO: `/v3/projects/90633921fc274ec9b7f567af4ceb50af` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:01.714584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90633921fc274ec9b7f567af4ceb50af)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:01.715129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:01.854636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:01.862613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90633921fc274ec9b7f567af4ceb50af', 'request_id': 'req-c7235f9b-af6d-44de-8394-8049d8308d8e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.868529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90633921fc274ec9b7f567af4ceb50af', 'request_id': 'req-c7235f9b-af6d-44de-8394-8049d8308d8e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.875487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90633921fc274ec9b7f567af4ceb50af', 'request_id': 'req-c7235f9b-af6d-44de-8394-8049d8308d8e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.894969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:01.950437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] The token cache is being invalidate because project 90633921fc274ec9b7f567af4ceb50af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:01.950665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.951307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:01.952809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 830/1631] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:01 2022] DELETE /identity/v3/projects/90633921fc274ec9b7f567af4ceb50af => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:01.957205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6974d73-c9eb-4001-b71a-3b9a61a64da0 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:01.966666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6974d73-c9eb-4001-b71a-3b9a61a64da0 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:02.025228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:02.025873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:02.026191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:02.026471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] PATH_INFO: `/v3/users/41b16402a33642a599ec855fd2774edf` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:02.027618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b16402a33642a599ec855fd2774edf)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:02.029551 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:02.294369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b16402a33642a599ec855fd2774edf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:02.306184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b16402a33642a599ec855fd2774edf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:02.378510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b16402a33642a599ec855fd2774edf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:02.379015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:02.379449 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b16402a33642a599ec855fd2774edf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:02.387010 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3de2be5b-20bb-4537-82c3-6edffd3441f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b16402a33642a599ec855fd2774edf'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:02.393660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 802/1632] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:01 2022] DELETE /identity/v3/users/41b16402a33642a599ec855fd2774edf => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:02.454631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:02.800608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:02.802204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:02.817198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7235f9b-af6d-44de-8394-8049d8308d8e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:02.875645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:02.876560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:02.876882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:02.877420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] PATH_INFO: `/v3/projects/d0b7426c18fd4199890871d0dc953b1f` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:02.878941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b7426c18fd4199890871d0dc953b1f)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:02.880887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:02.917227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:02.968579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b7426c18fd4199890871d0dc953b1f', 'request_id': 'req-f0197935-884c-4ddf-9615-aac0750e667f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:02.974625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b7426c18fd4199890871d0dc953b1f', 'request_id': 'req-f0197935-884c-4ddf-9615-aac0750e667f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:02.986669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b7426c18fd4199890871d0dc953b1f', 'request_id': 'req-f0197935-884c-4ddf-9615-aac0750e667f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:03.111738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:03.203503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:03.208646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] The token cache is being invalidate because project d0b7426c18fd4199890871d0dc953b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:03.208937 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:03.209560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f0197935-884c-4ddf-9615-aac0750e667f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:03.211116 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 831/1633] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:02 2022] DELETE /identity/v3/projects/d0b7426c18fd4199890871d0dc953b1f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:03.217059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72858041-f1b8-4f54-a170-2c0e184fa8cd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:03.217400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72858041-f1b8-4f54-a170-2c0e184fa8cd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:03.217678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72858041-f1b8-4f54-a170-2c0e184fa8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:03.237373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-72858041-f1b8-4f54-a170-2c0e184fa8cd None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:03.238537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-72858041-f1b8-4f54-a170-2c0e184fa8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:03.300099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 803/1634] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:03.308443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa58af5-5e3a-41bf-8339-43821e68be7a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:03.308733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa58af5-5e3a-41bf-8339-43821e68be7a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:03.309036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa58af5-5e3a-41bf-8339-43821e68be7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:03.330663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-8fa58af5-5e3a-41bf-8339-43821e68be7a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:03.331870 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8fa58af5-5e3a-41bf-8339-43821e68be7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:03.339855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 832/1635] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:03.344143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72858041-f1b8-4f54-a170-2c0e184fa8cd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:03.354486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72858041-f1b8-4f54-a170-2c0e184fa8cd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:03.359211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:03.359818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:03.360098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:03.360373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:03.361044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:03.361349 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:03.363588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:03.375266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 804/1636] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:03.380481 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa58af5-5e3a-41bf-8339-43821e68be7a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:03.390224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa58af5-5e3a-41bf-8339-43821e68be7a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:03.394878 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2969dff-b45b-4e5f-83ff-be2f371526a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:03.395477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969dff-b45b-4e5f-83ff-be2f371526a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:03.395750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969dff-b45b-4e5f-83ff-be2f371526a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:03.396046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969dff-b45b-4e5f-83ff-be2f371526a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:03.396677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2969dff-b45b-4e5f-83ff-be2f371526a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:03.398599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2969dff-b45b-4e5f-83ff-be2f371526a3 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:03.592898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 833/1637] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:03 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:03.621538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:03.632566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ead240-b8ca-43fc-80ad-637f2d8d3f0f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:03.689903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7853580-bc27-46ea-bbe5-34c3ef014c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:03.851509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7853580-bc27-46ea-bbe5-34c3ef014c47 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:03.851509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7853580-bc27-46ea-bbe5-34c3ef014c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:03.851509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7853580-bc27-46ea-bbe5-34c3ef014c47 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:03.851509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7853580-bc27-46ea-bbe5-34c3ef014c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:03.851509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7853580-bc27-46ea-bbe5-34c3ef014c47 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:03.974378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 805/1638] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:03 2022] POST /identity/v3/users => generated 288 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:03.978328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2969dff-b45b-4e5f-83ff-be2f371526a3 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:03.988320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2969dff-b45b-4e5f-83ff-be2f371526a3 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:03.993205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4a7b9-670c-45a2-9357-b6e514ff2112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:03.993795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4a7b9-670c-45a2-9357-b6e514ff2112 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:03.994114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4a7b9-670c-45a2-9357-b6e514ff2112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:03.994385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4a7b9-670c-45a2-9357-b6e514ff2112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:03.995016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4a7b9-670c-45a2-9357-b6e514ff2112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:03.996600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4a7b9-670c-45a2-9357-b6e514ff2112 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:04.009010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 834/1639] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:04.059793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7853580-bc27-46ea-bbe5-34c3ef014c47 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:04.075229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7853580-bc27-46ea-bbe5-34c3ef014c47 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:04.081752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608ba5c-d422-46fe-a694-dcdce75321ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:04.082529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608ba5c-d422-46fe-a694-dcdce75321ea admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:04.082861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608ba5c-d422-46fe-a694-dcdce75321ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:04.083217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608ba5c-d422-46fe-a694-dcdce75321ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:04.083990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608ba5c-d422-46fe-a694-dcdce75321ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:04.085917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608ba5c-d422-46fe-a694-dcdce75321ea admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:04.101205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 806/1640] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:04.111202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4a7b9-670c-45a2-9357-b6e514ff2112 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:04.122478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4a7b9-670c-45a2-9357-b6e514ff2112 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:04.127781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:04.128436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:04.128733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:04.129051 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] PATH_INFO: `/v3/projects/c22e6d2682174c1186798d1df5fdc513/users/6ff4c91600a949839f346e0a6308260c/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:04.150395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22e6d2682174c1186798d1df5fdc513, user_id=6ff4c91600a949839f346e0a6308260c, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:04.153266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:04.586569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 835/1641] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:04 2022] PUT /identity/v3/projects/c22e6d2682174c1186798d1df5fdc513/users/6ff4c91600a949839f346e0a6308260c/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:04.598896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af3d54-046e-4966-a39b-8a9dbe886c94 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:04.599186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af3d54-046e-4966-a39b-8a9dbe886c94 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:04.599462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af3d54-046e-4966-a39b-8a9dbe886c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:04.626243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:04.631107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-44af3d54-046e-4966-a39b-8a9dbe886c94 None None] MFA Rules not processed for user `6ff4c91600a949839f346e0a6308260c`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:04.631107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-44af3d54-046e-4966-a39b-8a9dbe886c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:04.698233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 807/1642] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:04.717418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:04.770865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:04.772913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:04.828881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd26d0-a600-4fd5-82cd-6033a58b9c61 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:04.832698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f808a1-b731-40e2-a817-f712931bbc7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:04.833300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f808a1-b731-40e2-a817-f712931bbc7b service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:04.833578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f808a1-b731-40e2-a817-f712931bbc7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:04.833851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f808a1-b731-40e2-a817-f712931bbc7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:04.842355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f808a1-b731-40e2-a817-f712931bbc7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:04.843780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f808a1-b731-40e2-a817-f712931bbc7b service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:04.853306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 836/1643] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:05.077667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:05.512135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:05.875611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:05.963036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:06.024917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af3d54-046e-4966-a39b-8a9dbe886c94 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:06.026618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-44af3d54-046e-4966-a39b-8a9dbe886c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:06.094684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af3d54-046e-4966-a39b-8a9dbe886c94 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:06.099329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:06.099970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:06.100250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:06.100525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] PATH_INFO: `/v3/users/f6738fcb683c4230954259fd6615750a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:06.101652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6738fcb683c4230954259fd6615750a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:06.103605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:06.299014 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6738fcb683c4230954259fd6615750a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:06.312065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6738fcb683c4230954259fd6615750a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:06.334457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:06.379718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6738fcb683c4230954259fd6615750a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:06.380394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6738fcb683c4230954259fd6615750a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:06.387578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:06.389573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6738fcb683c4230954259fd6615750a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:06.395434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 808/1644] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:06 2022] DELETE /identity/v3/users/f6738fcb683c4230954259fd6615750a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:06.399868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f54c4e-f339-44c7-8d72-4959bc515bf3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:06.400154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f54c4e-f339-44c7-8d72-4959bc515bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:06.400443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f54c4e-f339-44c7-8d72-4959bc515bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:06.418822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b1f54c4e-f339-44c7-8d72-4959bc515bf3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:06.419997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b1f54c4e-f339-44c7-8d72-4959bc515bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:06.477952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 837/1645] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:06.488364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:06.489773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:06.548505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a6bec-cab5-466d-be5e-f6d8778adb14 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:06.552828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5388-5287-4a31-834f-adcaee990cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:06.553286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5388-5287-4a31-834f-adcaee990cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:06.553569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5388-5287-4a31-834f-adcaee990cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:06.553841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb5388-5287-4a31-834f-adcaee990cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:06.557524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:06.563063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb5388-5287-4a31-834f-adcaee990cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:06.564508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb5388-5287-4a31-834f-adcaee990cd4 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:06.574183 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 809/1646] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:06.717467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:06.882989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:06.928600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:06.958581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f54c4e-f339-44c7-8d72-4959bc515bf3 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:06.960073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b1f54c4e-f339-44c7-8d72-4959bc515bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:06.972871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f54c4e-f339-44c7-8d72-4959bc515bf3 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:06.978549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:06.978549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:06.978549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:06.978852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] PATH_INFO: `/v3/projects/6588debbf9374d8f97af3ce21f0be38d` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:06.979942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6588debbf9374d8f97af3ce21f0be38d)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:06.981858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:07.095036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6588debbf9374d8f97af3ce21f0be38d', 'request_id': 'req-45b9aa35-5b55-4b13-86e1-463e12bea791'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.101047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6588debbf9374d8f97af3ce21f0be38d', 'request_id': 'req-45b9aa35-5b55-4b13-86e1-463e12bea791'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.107912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6588debbf9374d8f97af3ce21f0be38d', 'request_id': 'req-45b9aa35-5b55-4b13-86e1-463e12bea791'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.184816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] The token cache is being invalidate because project 6588debbf9374d8f97af3ce21f0be38d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:07.185098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.185712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.187259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 838/1647] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:06 2022] DELETE /identity/v3/projects/6588debbf9374d8f97af3ce21f0be38d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:07.191288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5388-5287-4a31-834f-adcaee990cd4 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:07.201100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb5388-5287-4a31-834f-adcaee990cd4 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:07.260573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:07.261254 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:07.261539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:07.261812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] PATH_INFO: `/v3/users/966f3a1abf7f4a60933b4c1cb529d26e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:07.263190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=966f3a1abf7f4a60933b4c1cb529d26e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:07.265123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:07.284078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:07.461192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966f3a1abf7f4a60933b4c1cb529d26e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.474052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966f3a1abf7f4a60933b4c1cb529d26e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.543713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966f3a1abf7f4a60933b4c1cb529d26e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.544339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966f3a1abf7f4a60933b4c1cb529d26e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.551426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e990d904-8a8b-4b73-a1c4-37d171936b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966f3a1abf7f4a60933b4c1cb529d26e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:07.557403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 810/1648] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:07 2022] DELETE /identity/v3/users/966f3a1abf7f4a60933b4c1cb529d26e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:07.854142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:07.994882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:07.996527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:08.010789 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9aa35-5b55-4b13-86e1-463e12bea791 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:08.095969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:08.097113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.097439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.097712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] PATH_INFO: `/v3/projects/121d1876ac644b5ca30b68508274ea7f` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.098879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121d1876ac644b5ca30b68508274ea7f)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:08.100908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:08.212488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121d1876ac644b5ca30b68508274ea7f', 'request_id': 'req-2aabf22a-8b4a-4858-837c-3f02eefb6e16'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:08.218865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121d1876ac644b5ca30b68508274ea7f', 'request_id': 'req-2aabf22a-8b4a-4858-837c-3f02eefb6e16'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:08.227074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121d1876ac644b5ca30b68508274ea7f', 'request_id': 'req-2aabf22a-8b4a-4858-837c-3f02eefb6e16'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:08.235843 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:08.299941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] The token cache is being invalidate because project 121d1876ac644b5ca30b68508274ea7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:08.300359 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:08.300997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2aabf22a-8b4a-4858-837c-3f02eefb6e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:08.302656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 839/1649] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:07 2022] DELETE /identity/v3/projects/121d1876ac644b5ca30b68508274ea7f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:08.308201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3220da-9878-47aa-9bc6-9d7558ad781a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.308585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3220da-9878-47aa-9bc6-9d7558ad781a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.308891 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3220da-9878-47aa-9bc6-9d7558ad781a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.328192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4d3220da-9878-47aa-9bc6-9d7558ad781a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:08.329387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4d3220da-9878-47aa-9bc6-9d7558ad781a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:08.395676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 811/1650] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:08.406979 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9966cfa-42db-406b-8979-245c907696d0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.407265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9966cfa-42db-406b-8979-245c907696d0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.407542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9966cfa-42db-406b-8979-245c907696d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.432213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a9966cfa-42db-406b-8979-245c907696d0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:08.433504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a9966cfa-42db-406b-8979-245c907696d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:08.441996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 840/1651] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:08.446177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3220da-9878-47aa-9bc6-9d7558ad781a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:08.457225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3220da-9878-47aa-9bc6-9d7558ad781a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:08.461978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:08.462600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.462933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.463201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.464009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:08.464622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:08.466262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:08.478996 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 812/1652] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:08.483342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9966cfa-42db-406b-8979-245c907696d0 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:08.492950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9966cfa-42db-406b-8979-245c907696d0 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:08.497543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0f91-6785-4221-ba5f-efccdace95df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:08.498172 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0f91-6785-4221-ba5f-efccdace95df admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.498451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0f91-6785-4221-ba5f-efccdace95df admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.498728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb0f91-6785-4221-ba5f-efccdace95df admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.499391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb0f91-6785-4221-ba5f-efccdace95df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:08.501322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb0f91-6785-4221-ba5f-efccdace95df admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:08.643626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 841/1653] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:08 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:08.647769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:08.657765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d5a3-fd60-46c7-a09e-a98f3089b171 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:08.723376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fa6c3-a420-4a4b-84b8-8f73d3ab48bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:08.724017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fa6c3-a420-4a4b-84b8-8f73d3ab48bf admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.724300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fa6c3-a420-4a4b-84b8-8f73d3ab48bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.724935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fa6c3-a420-4a4b-84b8-8f73d3ab48bf admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.725559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fa6c3-a420-4a4b-84b8-8f73d3ab48bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:08.727733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fa6c3-a420-4a4b-84b8-8f73d3ab48bf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:08.873108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 813/1654] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:08 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:08.880590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0f91-6785-4221-ba5f-efccdace95df admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:08.900714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb0f91-6785-4221-ba5f-efccdace95df admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:08.905810 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187fda-e1cb-49de-aa0f-2a632655dfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:08.906449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187fda-e1cb-49de-aa0f-2a632655dfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.906726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187fda-e1cb-49de-aa0f-2a632655dfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.907060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187fda-e1cb-49de-aa0f-2a632655dfd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.907701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187fda-e1cb-49de-aa0f-2a632655dfd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:08.909571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187fda-e1cb-49de-aa0f-2a632655dfd9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:08.925133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 842/1655] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:08.925799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fa6c3-a420-4a4b-84b8-8f73d3ab48bf admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:08.936659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fa6c3-a420-4a4b-84b8-8f73d3ab48bf admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:08.941952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9aa76-f515-43cc-b3b0-cd6d25c037fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:08.942608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9aa76-f515-43cc-b3b0-cd6d25c037fe admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.942885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9aa76-f515-43cc-b3b0-cd6d25c037fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.943154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9aa76-f515-43cc-b3b0-cd6d25c037fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.943790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9aa76-f515-43cc-b3b0-cd6d25c037fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:08.945356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9aa76-f515-43cc-b3b0-cd6d25c037fe admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:08.956148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 814/1656] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:08.960853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187fda-e1cb-49de-aa0f-2a632655dfd9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:08.970730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187fda-e1cb-49de-aa0f-2a632655dfd9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:08.975512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16c77e-3939-4a78-9505-35c8d9eeb365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:08.976106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16c77e-3939-4a78-9505-35c8d9eeb365 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.976381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16c77e-3939-4a78-9505-35c8d9eeb365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.976651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16c77e-3939-4a78-9505-35c8d9eeb365 admin admin] PATH_INFO: `/v3/projects/b2ca25970dd640e4939d88a06a74b795/users/d64cc730513c46e6b0133b284dfdd0a9/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.995129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16c77e-3939-4a78-9505-35c8d9eeb365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ca25970dd640e4939d88a06a74b795, user_id=d64cc730513c46e6b0133b284dfdd0a9, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:09.001069 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16c77e-3939-4a78-9505-35c8d9eeb365 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:09.099017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 843/1657] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:08 2022] PUT /identity/v3/projects/b2ca25970dd640e4939d88a06a74b795/users/d64cc730513c46e6b0133b284dfdd0a9/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:09.109509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9083-3d9c-4596-9060-805b19e32835 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:09.109800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9083-3d9c-4596-9060-805b19e32835 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:09.110106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9083-3d9c-4596-9060-805b19e32835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:09.138653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fa0f9083-3d9c-4596-9060-805b19e32835 None None] MFA Rules not processed for user `d64cc730513c46e6b0133b284dfdd0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:09.139832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fa0f9083-3d9c-4596-9060-805b19e32835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:09.206154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 815/1658] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:09.210837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16c77e-3939-4a78-9505-35c8d9eeb365 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:09.286540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16c77e-3939-4a78-9505-35c8d9eeb365 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:09.292766 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b666c-e3ea-4d1d-9130-6ff28a75b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:09.293056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b666c-e3ea-4d1d-9130-6ff28a75b30c admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:09.293386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b666c-e3ea-4d1d-9130-6ff28a75b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:09.293703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b666c-e3ea-4d1d-9130-6ff28a75b30c admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:09.294379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b666c-e3ea-4d1d-9130-6ff28a75b30c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:09.296262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b666c-e3ea-4d1d-9130-6ff28a75b30c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:09.439433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 844/1659] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:09 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:09.444089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f9083-3d9c-4596-9060-805b19e32835 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:09.455178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f9083-3d9c-4596-9060-805b19e32835 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:09.513032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990e6a-6d76-4e3e-81e4-3fe43907ef59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:09.513641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990e6a-6d76-4e3e-81e4-3fe43907ef59 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:09.513959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990e6a-6d76-4e3e-81e4-3fe43907ef59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:09.515508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990e6a-6d76-4e3e-81e4-3fe43907ef59 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:09.515508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c990e6a-6d76-4e3e-81e4-3fe43907ef59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:09.517005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c990e6a-6d76-4e3e-81e4-3fe43907ef59 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:09.623492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:09.641892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 816/1660] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:09 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:09.645870 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b666c-e3ea-4d1d-9130-6ff28a75b30c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:09.657388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b666c-e3ea-4d1d-9130-6ff28a75b30c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:09.663970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d142-aba1-43cc-898b-4dede057d999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:09.664825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d142-aba1-43cc-898b-4dede057d999 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:09.665562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d142-aba1-43cc-898b-4dede057d999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:09.665562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d142-aba1-43cc-898b-4dede057d999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:09.666293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07d142-aba1-43cc-898b-4dede057d999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:09.668090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07d142-aba1-43cc-898b-4dede057d999 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:09.681776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 845/1661] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:09.685647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990e6a-6d76-4e3e-81e4-3fe43907ef59 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:09.697217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990e6a-6d76-4e3e-81e4-3fe43907ef59 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:09.714095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c74649-7d2a-4ffd-9567-f67907333b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:09.716802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c74649-7d2a-4ffd-9567-f67907333b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:09.716802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c74649-7d2a-4ffd-9567-f67907333b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:09.716802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c74649-7d2a-4ffd-9567-f67907333b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:09.716802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c74649-7d2a-4ffd-9567-f67907333b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:09.718220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c74649-7d2a-4ffd-9567-f67907333b2c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:09.735887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 817/1662] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:09.737302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d142-aba1-43cc-898b-4dede057d999 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:09.749077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d142-aba1-43cc-898b-4dede057d999 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:09.754365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:09.754854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:09.755133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:09.755402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] PATH_INFO: `/v3/projects/9eb6c88e7c924b4f82244cb74465475d/users/2befdeb24cea4c82ba66ad889bbecd67/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:09.770984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb6c88e7c924b4f82244cb74465475d, user_id=2befdeb24cea4c82ba66ad889bbecd67, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:09.773788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:09.872228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 846/1663] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:09 2022] PUT /identity/v3/projects/9eb6c88e7c924b4f82244cb74465475d/users/2befdeb24cea4c82ba66ad889bbecd67/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:09.884094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1cc58-b121-4730-999a-86fbf2d1348c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:09.884386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1cc58-b121-4730-999a-86fbf2d1348c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:09.884672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1cc58-b121-4730-999a-86fbf2d1348c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:09.918115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-55c1cc58-b121-4730-999a-86fbf2d1348c None None] MFA Rules not processed for user `2befdeb24cea4c82ba66ad889bbecd67`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:09.919431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-55c1cc58-b121-4730-999a-86fbf2d1348c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:09.948948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:09.950745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:10.001970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:10.002798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 818/1664] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:10.007628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1cc58-b121-4730-999a-86fbf2d1348c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.031256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e87ff-aca0-4c8b-8f29-497ac5fc8269 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.042633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.043613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.044354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.044986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] PATH_INFO: `/v3/users/6ec8b89213554c65a6d534428094d569` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.046966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec8b89213554c65a6d534428094d569)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.049847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.089084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1cc58-b121-4730-999a-86fbf2d1348c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.093912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6666c-b577-4c83-880a-a74353b33753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.098525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6666c-b577-4c83-880a-a74353b33753 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.098525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6666c-b577-4c83-880a-a74353b33753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.098525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6666c-b577-4c83-880a-a74353b33753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.098525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6666c-b577-4c83-880a-a74353b33753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.098525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6666c-b577-4c83-880a-a74353b33753 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.204917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 819/1665] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:10 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:10.209526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6666c-b577-4c83-880a-a74353b33753 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.279274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6666c-b577-4c83-880a-a74353b33753 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.290156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec8b89213554c65a6d534428094d569'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.290618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d597ad9-12da-4fb7-8562-98966af814b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.291316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597ad9-12da-4fb7-8562-98966af814b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.291532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597ad9-12da-4fb7-8562-98966af814b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.291965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597ad9-12da-4fb7-8562-98966af814b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.292485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d597ad9-12da-4fb7-8562-98966af814b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.294678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d597ad9-12da-4fb7-8562-98966af814b6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.309135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec8b89213554c65a6d534428094d569'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.309922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec8b89213554c65a6d534428094d569'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.368506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec8b89213554c65a6d534428094d569'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.375988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec8b89213554c65a6d534428094d569'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.382403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 847/1666] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:09 2022] DELETE /identity/v3/users/6ec8b89213554c65a6d534428094d569 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:10.383184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.384471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:10.445728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 820/1667] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:10 2022] POST /identity/v3/users => generated 298 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:10.445728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf16e-c581-41b8-9a88-8e2d41c3b1e3 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.445728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf16e-c581-41b8-9a88-8e2d41c3b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.445728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf16e-c581-41b8-9a88-8e2d41c3b1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.447095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c9aa9-3c97-469b-a38f-812a4996c95e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.451780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.452488 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.452720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.452997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] PATH_INFO: `/v3/users/6ff4c91600a949839f346e0a6308260c` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.454451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff4c91600a949839f346e0a6308260c)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.457074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.465108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a79bf16e-c581-41b8-9a88-8e2d41c3b1e3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:10.466384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a79bf16e-c581-41b8-9a88-8e2d41c3b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:10.474557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 821/1668] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:10.477690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bf16e-c581-41b8-9a88-8e2d41c3b1e3 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.477690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a79bf16e-c581-41b8-9a88-8e2d41c3b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:10.492277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bf16e-c581-41b8-9a88-8e2d41c3b1e3 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.497142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.497748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.498062 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.498338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.498979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.500567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.511269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 822/1669] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:10.512144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.513900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:10.569374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a819a-a17c-4313-b204-2de0bd9ba914 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.573024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59211f06-a066-44f5-a9f9-c3e0ecbb7894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.573616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59211f06-a066-44f5-a9f9-c3e0ecbb7894 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.573957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59211f06-a066-44f5-a9f9-c3e0ecbb7894 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.574249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59211f06-a066-44f5-a9f9-c3e0ecbb7894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.583656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59211f06-a066-44f5-a9f9-c3e0ecbb7894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.586469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59211f06-a066-44f5-a9f9-c3e0ecbb7894 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.596122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 823/1670] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:10.596915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59211f06-a066-44f5-a9f9-c3e0ecbb7894 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.605774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59211f06-a066-44f5-a9f9-c3e0ecbb7894 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.610396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.611034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.611317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.611595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] PATH_INFO: `/v3/projects/c4c203138528419ba9515e3f48095ea5/users/7d5b86d11ed340a8b06f89b5b44326c6/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.629241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c203138528419ba9515e3f48095ea5, user_id=7d5b86d11ed340a8b06f89b5b44326c6, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.632033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.658327 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff4c91600a949839f346e0a6308260c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.675602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff4c91600a949839f346e0a6308260c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.676356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff4c91600a949839f346e0a6308260c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.741478 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 824/1671] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:10 2022] PUT /identity/v3/projects/c4c203138528419ba9515e3f48095ea5/users/7d5b86d11ed340a8b06f89b5b44326c6/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:10.747597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.749393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:10.764032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff4c91600a949839f346e0a6308260c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.775935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fcd5fe89-5ec3-4071-8df1-daecf29644fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff4c91600a949839f346e0a6308260c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:10.789111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 848/1672] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:10 2022] DELETE /identity/v3/users/6ff4c91600a949839f346e0a6308260c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:10.790972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9b19-827b-4eef-b0a4-b64037c52787 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.791263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9b19-827b-4eef-b0a4-b64037c52787 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.792574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9b19-827b-4eef-b0a4-b64037c52787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.821820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-f04b9b19-827b-4eef-b0a4-b64037c52787 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:10.823442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f04b9b19-827b-4eef-b0a4-b64037c52787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:10.857791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dddee-c2c2-4ec6-8ff3-653f100c283e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.905850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 849/1673] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:10.920004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9b19-827b-4eef-b0a4-b64037c52787 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.922510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f04b9b19-827b-4eef-b0a4-b64037c52787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:10.942480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06115d-6b49-4675-b2e4-98c8584660af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.943310 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06115d-6b49-4675-b2e4-98c8584660af admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.943781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06115d-6b49-4675-b2e4-98c8584660af admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.944427 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06115d-6b49-4675-b2e4-98c8584660af admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.945177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06115d-6b49-4675-b2e4-98c8584660af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.947609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06115d-6b49-4675-b2e4-98c8584660af admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.962055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 825/1674] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:10.965832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06115d-6b49-4675-b2e4-98c8584660af admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:10.974350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06115d-6b49-4675-b2e4-98c8584660af admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:10.978876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:10.979111 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b432ae-2371-482f-bd11-981922a1e9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:10.979792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b432ae-2371-482f-bd11-981922a1e9e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:10.980053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b432ae-2371-482f-bd11-981922a1e9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:10.980322 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b432ae-2371-482f-bd11-981922a1e9e0 admin admin] PATH_INFO: `/v3/projects/c4c203138528419ba9515e3f48095ea5/users/7d5b86d11ed340a8b06f89b5b44326c6/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:10.982393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b432ae-2371-482f-bd11-981922a1e9e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c203138528419ba9515e3f48095ea5, user_id=7d5b86d11ed340a8b06f89b5b44326c6, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:10.985009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b432ae-2371-482f-bd11-981922a1e9e0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:10.996065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 826/1675] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:10 2022] PUT /identity/v3/projects/c4c203138528419ba9515e3f48095ea5/users/7d5b86d11ed340a8b06f89b5b44326c6/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:10.997033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9b19-827b-4eef-b0a4-b64037c52787 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:11.006778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276498dd-950c-489c-a0bf-7e9115c39ce4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:11.007131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276498dd-950c-489c-a0bf-7e9115c39ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:11.007414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276498dd-950c-489c-a0bf-7e9115c39ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:11.035855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-276498dd-950c-489c-a0bf-7e9115c39ce4 None None] MFA Rules not processed for user `7d5b86d11ed340a8b06f89b5b44326c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:11.037080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-276498dd-950c-489c-a0bf-7e9115c39ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:11.045548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:11.046182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:11.046469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:11.046739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:11.117442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:11.122033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 827/1676] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:11.122294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:11.122383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276498dd-950c-489c-a0bf-7e9115c39ce4 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:11.122383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-276498dd-950c-489c-a0bf-7e9115c39ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:11.129665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:11.130916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 850/1677] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:11.131828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:11.144367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:11.152689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276498dd-950c-489c-a0bf-7e9115c39ce4 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:11.157796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:11.158719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:11.159059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:11.159995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] PATH_INFO: `/v3/projects/71a87986b04c430d970420b45231061f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:11.161983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a87986b04c430d970420b45231061f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:11.164107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:11.208554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a83ba-b824-4832-a43c-5e0b03a43713 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:11.212533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d5cd1-7959-4d89-ac91-946d85c5ef26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:11.213068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5cd1-7959-4d89-ac91-946d85c5ef26 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:11.213335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5cd1-7959-4d89-ac91-946d85c5ef26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:11.214205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5cd1-7959-4d89-ac91-946d85c5ef26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:11.224140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d5cd1-7959-4d89-ac91-946d85c5ef26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:11.225606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d5cd1-7959-4d89-ac91-946d85c5ef26 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:11.235439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 851/1678] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:11.328373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a87986b04c430d970420b45231061f', 'request_id': 'req-79764b42-70af-43fc-900c-78e06ba2dde2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.337352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a87986b04c430d970420b45231061f', 'request_id': 'req-79764b42-70af-43fc-900c-78e06ba2dde2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.355348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a87986b04c430d970420b45231061f', 'request_id': 'req-79764b42-70af-43fc-900c-78e06ba2dde2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.447048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] The token cache is being invalidate because project 71a87986b04c430d970420b45231061f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:11.447416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.448049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.449789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 828/1679] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:11 2022] DELETE /identity/v3/projects/71a87986b04c430d970420b45231061f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:11.453743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d5cd1-7959-4d89-ac91-946d85c5ef26 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:11.465346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d5cd1-7959-4d89-ac91-946d85c5ef26 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:11.532283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:11.533075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:11.533430 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:11.533774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] PATH_INFO: `/v3/users/327ff41d31754fd8930e7e8892a96a2e` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:11.535555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327ff41d31754fd8930e7e8892a96a2e)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:11.537711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:11.676401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:11.678153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:11.691309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79764b42-70af-43fc-900c-78e06ba2dde2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:11.697136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:11.697819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:11.698129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:11.698420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] PATH_INFO: `/v3/projects/c22e6d2682174c1186798d1df5fdc513` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:11.699471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22e6d2682174c1186798d1df5fdc513)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:11.701641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:11.775381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327ff41d31754fd8930e7e8892a96a2e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.788487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327ff41d31754fd8930e7e8892a96a2e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.789322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327ff41d31754fd8930e7e8892a96a2e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.843603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22e6d2682174c1186798d1df5fdc513', 'request_id': 'req-9714f442-ae6f-4db6-8767-87755fa22336'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.849203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22e6d2682174c1186798d1df5fdc513', 'request_id': 'req-9714f442-ae6f-4db6-8767-87755fa22336'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.856100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327ff41d31754fd8930e7e8892a96a2e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.856406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22e6d2682174c1186798d1df5fdc513', 'request_id': 'req-9714f442-ae6f-4db6-8767-87755fa22336'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.865194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-977326e8-b96f-4353-86a8-73f72f83cd52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327ff41d31754fd8930e7e8892a96a2e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.871473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 852/1680] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:11 2022] DELETE /identity/v3/users/327ff41d31754fd8930e7e8892a96a2e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:11.948017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] The token cache is being invalidate because project c22e6d2682174c1186798d1df5fdc513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:11.948457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.949748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9714f442-ae6f-4db6-8767-87755fa22336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:11.951866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 829/1681] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:11 2022] DELETE /identity/v3/projects/c22e6d2682174c1186798d1df5fdc513 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:11.957761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3046-5f92-4914-89fb-ab71bf2f72ab None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:11.958315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3046-5f92-4914-89fb-ab71bf2f72ab None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:11.960315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba3046-5f92-4914-89fb-ab71bf2f72ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:11.984278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b2ba3046-5f92-4914-89fb-ab71bf2f72ab None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:11.985582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b2ba3046-5f92-4914-89fb-ab71bf2f72ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:12.046664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 853/1682] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:12.054402 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f024b1-fa5c-4eb4-b7d3-44397ed4cafc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.054692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f024b1-fa5c-4eb4-b7d3-44397ed4cafc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.054975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f024b1-fa5c-4eb4-b7d3-44397ed4cafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.077436 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-84f024b1-fa5c-4eb4-b7d3-44397ed4cafc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:12.078853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-84f024b1-fa5c-4eb4-b7d3-44397ed4cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:12.088198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 830/1683] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:12.096048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba3046-5f92-4914-89fb-ab71bf2f72ab None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.108634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba3046-5f92-4914-89fb-ab71bf2f72ab None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.115443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.116059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.116629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.116868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.117646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:12.117978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.122793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.135617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 854/1684] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:12.141309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f024b1-fa5c-4eb4-b7d3-44397ed4cafc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.151516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f024b1-fa5c-4eb4-b7d3-44397ed4cafc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.156389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.157001 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.157295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.157557 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.158230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.160175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.192384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:12.261051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 831/1685] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:12 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:12.262280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.272783 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf59e9-e665-4ea7-85ba-90415a6c046f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.343797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.344451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.344734 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.345065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.345702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.347682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.351882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:12.430451 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.432198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:12.474223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 855/1686] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:12 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:12.474223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.474223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:12.486614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29eb1e-6111-4df9-a96a-b221898f1f70 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.492443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.493172 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.493547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.493828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] PATH_INFO: `/v3/projects/c68b8d48007640e58ed94846648b3eec` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.494992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68b8d48007640e58ed94846648b3eec)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.496660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18434cbe-0932-410a-91fa-3a4daf4464d6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.496887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.500349 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.501144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.502464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.502464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.504136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:12.563295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.564738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.574568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 832/1687] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:12.577108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.587999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e47156-234c-445e-a392-6ed2b9cd57f4 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.593198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616123f-c8fe-425b-8c99-a30d25d96e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.594477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616123f-c8fe-425b-8c99-a30d25d96e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.594762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616123f-c8fe-425b-8c99-a30d25d96e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.595037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616123f-c8fe-425b-8c99-a30d25d96e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.595688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616123f-c8fe-425b-8c99-a30d25d96e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.597814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616123f-c8fe-425b-8c99-a30d25d96e0a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.609645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 833/1688] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:12.614476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616123f-c8fe-425b-8c99-a30d25d96e0a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.626159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616123f-c8fe-425b-8c99-a30d25d96e0a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.639815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68b8d48007640e58ed94846648b3eec', 'request_id': 'req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:12.652262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68b8d48007640e58ed94846648b3eec', 'request_id': 'req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:12.662782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68b8d48007640e58ed94846648b3eec', 'request_id': 'req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:12.702630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dbe45-da68-44a5-b883-0f8889750315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.703349 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dbe45-da68-44a5-b883-0f8889750315 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.703762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dbe45-da68-44a5-b883-0f8889750315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.704064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dbe45-da68-44a5-b883-0f8889750315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.704882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dbe45-da68-44a5-b883-0f8889750315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.706837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dbe45-da68-44a5-b883-0f8889750315 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.717967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 834/1689] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:12.722799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dbe45-da68-44a5-b883-0f8889750315 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.736756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dbe45-da68-44a5-b883-0f8889750315 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.741673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] The token cache is being invalidate because project c68b8d48007640e58ed94846648b3eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:12.743813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:12.743813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a7e084d3-3ec4-4589-bf2b-c9b8e1681672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:12.745838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 856/1690] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:12 2022] DELETE /identity/v3/projects/c68b8d48007640e58ed94846648b3eec => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:12.751761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52bd0d-d9a1-4e67-a55e-5e171fc9bc2b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.751761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52bd0d-d9a1-4e67-a55e-5e171fc9bc2b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.751761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52bd0d-d9a1-4e67-a55e-5e171fc9bc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.752125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9297e1-3c63-4865-97cc-4e74ab6cf6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.752399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9297e1-3c63-4865-97cc-4e74ab6cf6ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.752691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9297e1-3c63-4865-97cc-4e74ab6cf6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.752950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9297e1-3c63-4865-97cc-4e74ab6cf6ac admin admin] PATH_INFO: `/v3/projects/f431e712685e486eb5731f4b0b4b25a0/users/25f0393cb6cf4e93a4a8fb672ac03696/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.776835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-3d52bd0d-d9a1-4e67-a55e-5e171fc9bc2b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:12.778581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3d52bd0d-d9a1-4e67-a55e-5e171fc9bc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:12.783043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9297e1-3c63-4865-97cc-4e74ab6cf6ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f431e712685e486eb5731f4b0b4b25a0, user_id=25f0393cb6cf4e93a4a8fb672ac03696, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.786387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9297e1-3c63-4865-97cc-4e74ab6cf6ac admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.789504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 857/1691] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:12.797218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a962-34e5-4b4f-bc7e-2cf879658705 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.797888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a962-34e5-4b4f-bc7e-2cf879658705 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.797888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285a962-34e5-4b4f-bc7e-2cf879658705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.819454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d285a962-34e5-4b4f-bc7e-2cf879658705 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:12.820556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d285a962-34e5-4b4f-bc7e-2cf879658705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:12.829263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 858/1692] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:12.834161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285a962-34e5-4b4f-bc7e-2cf879658705 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.844003 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285a962-34e5-4b4f-bc7e-2cf879658705 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.849525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.850297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.850608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.850874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.851531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:12.851835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.853823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:12.867039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 835/1693] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:12 2022] PUT /identity/v3/projects/f431e712685e486eb5731f4b0b4b25a0/users/25f0393cb6cf4e93a4a8fb672ac03696/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:12.877159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5006-eafd-4e6e-8dbc-7a416b0e971e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.878071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5006-eafd-4e6e-8dbc-7a416b0e971e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.878071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc5006-eafd-4e6e-8dbc-7a416b0e971e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.885206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 859/1694] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:12.887046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:12.900097 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90272-4374-4527-94a5-e8821b3d383d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:12.906646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-83cc5006-eafd-4e6e-8dbc-7a416b0e971e None None] MFA Rules not processed for user `25f0393cb6cf4e93a4a8fb672ac03696`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:12.907861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-83cc5006-eafd-4e6e-8dbc-7a416b0e971e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:12.980187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200165db-c710-4beb-971c-902c7f356813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:12.980910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200165db-c710-4beb-971c-902c7f356813 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:12.981328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200165db-c710-4beb-971c-902c7f356813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:12.981636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200165db-c710-4beb-971c-902c7f356813 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:12.983133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200165db-c710-4beb-971c-902c7f356813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:12.984728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200165db-c710-4beb-971c-902c7f356813 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:13.008662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 836/1695] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:13.016139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc5006-eafd-4e6e-8dbc-7a416b0e971e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:13.017642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-83cc5006-eafd-4e6e-8dbc-7a416b0e971e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:13.112823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc5006-eafd-4e6e-8dbc-7a416b0e971e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:13.118082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:13.120786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e331b-7d29-444c-8049-568e841ad5b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cbcaeb843e470781661f00ec91bffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:13.121587 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e331b-7d29-444c-8049-568e841ad5b4 service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:13.121871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e331b-7d29-444c-8049-568e841ad5b4 service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:13.127379 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e331b-7d29-444c-8049-568e841ad5b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:13.128449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 860/1696] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:12 2022] POST /identity/v3/projects => generated 371 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:13.134828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200165db-c710-4beb-971c-902c7f356813 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:13.135663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e331b-7d29-444c-8049-568e841ad5b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:13.138330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e331b-7d29-444c-8049-568e841ad5b4 service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:13.165676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 837/1697] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:13.227509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200165db-c710-4beb-971c-902c7f356813 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:13.234290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dcf30-499b-4207-bec5-45d85f2d0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:13.235384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dcf30-499b-4207-bec5-45d85f2d0a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:13.235897 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dcf30-499b-4207-bec5-45d85f2d0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:13.236370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dcf30-499b-4207-bec5-45d85f2d0a83 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:13.237210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dcf30-499b-4207-bec5-45d85f2d0a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:13.239829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dcf30-499b-4207-bec5-45d85f2d0a83 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:13.373637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 861/1698] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:13 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:13.377920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e331b-7d29-444c-8049-568e841ad5b4 service glance] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:13.392049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:13.396685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e331b-7d29-444c-8049-568e841ad5b4 service glance] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:13.410966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c1b8a-0f3c-4be0-96b1-9b905d06fb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:13.411981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c1b8a-0f3c-4be0-96b1-9b905d06fb86 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:13.412470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c1b8a-0f3c-4be0-96b1-9b905d06fb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:13.412952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c1b8a-0f3c-4be0-96b1-9b905d06fb86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:13.414073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c1b8a-0f3c-4be0-96b1-9b905d06fb86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:13.417078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c1b8a-0f3c-4be0-96b1-9b905d06fb86 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:13.444525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 838/1699] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:13.448509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dcf30-499b-4207-bec5-45d85f2d0a83 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:13.463157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dcf30-499b-4207-bec5-45d85f2d0a83 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:13.469397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cb1b5-efcd-46d2-b417-7719b42938cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:13.470167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cb1b5-efcd-46d2-b417-7719b42938cd admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:13.470457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cb1b5-efcd-46d2-b417-7719b42938cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:13.470744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cb1b5-efcd-46d2-b417-7719b42938cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:13.471843 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cb1b5-efcd-46d2-b417-7719b42938cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:13.474096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cb1b5-efcd-46d2-b417-7719b42938cd admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:13.496477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 862/1700] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:13.500632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c1b8a-0f3c-4be0-96b1-9b905d06fb86 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:13.523258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c1b8a-0f3c-4be0-96b1-9b905d06fb86 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:13.534802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:13.536283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:13.536762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:13.537265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] PATH_INFO: `/v3/projects/56014854c0694a1498a455f64bb4a194/users/79d9d641e9bf4570afe85aeb9662f110/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:13.564407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=56014854c0694a1498a455f64bb4a194, user_id=79d9d641e9bf4570afe85aeb9662f110, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:13.571404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:13.695287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 839/1701] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:13 2022] PUT /identity/v3/projects/56014854c0694a1498a455f64bb4a194/users/79d9d641e9bf4570afe85aeb9662f110/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:13.700335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d3834-e3be-473f-9834-c2cd5274bcd4 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:13.700644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d3834-e3be-473f-9834-c2cd5274bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:13.700927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d3834-e3be-473f-9834-c2cd5274bcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:13.719596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-916d3834-e3be-473f-9834-c2cd5274bcd4 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:13.720809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-916d3834-e3be-473f-9834-c2cd5274bcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:13.779725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 863/1702] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:13.789530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:13.791405 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:13.860645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794d69f-6ae1-459c-9eff-f3dc9b2208dc admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:13.865243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7ee07-a141-454a-b755-efb4632c9cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:13.865883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7ee07-a141-454a-b755-efb4632c9cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:13.866192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7ee07-a141-454a-b755-efb4632c9cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:13.866727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7ee07-a141-454a-b755-efb4632c9cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:13.877535 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7ee07-a141-454a-b755-efb4632c9cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:13.879711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7ee07-a141-454a-b755-efb4632c9cb1 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:13.890874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 840/1703] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:14.329229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:15.231859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:15.484918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3a35-f4f5-4ca5-8d09-87ee87ca2203 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:15.485379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3a35-f4f5-4ca5-8d09-87ee87ca2203 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:15.485800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3a35-f4f5-4ca5-8d09-87ee87ca2203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:15.504727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:15.545263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c40a3a35-f4f5-4ca5-8d09-87ee87ca2203 None None] MFA Rules not processed for user `79d9d641e9bf4570afe85aeb9662f110`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:15.546538 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c40a3a35-f4f5-4ca5-8d09-87ee87ca2203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:15.652655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 864/1704] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:15.657297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7ee07-a141-454a-b755-efb4632c9cb1 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:15.670852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7ee07-a141-454a-b755-efb4632c9cb1 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:15.676764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb2bb2-65c5-41f3-b597-952cfefa9a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:15.677377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb2bb2-65c5-41f3-b597-952cfefa9a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:15.677652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb2bb2-65c5-41f3-b597-952cfefa9a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:15.677961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb2bb2-65c5-41f3-b597-952cfefa9a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:15.678601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb2bb2-65c5-41f3-b597-952cfefa9a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:15.680598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb2bb2-65c5-41f3-b597-952cfefa9a8b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:15.781904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 841/1705] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:15 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:15.786477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3a35-f4f5-4ca5-8d09-87ee87ca2203 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:15.797397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3a35-f4f5-4ca5-8d09-87ee87ca2203 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:15.857151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671dd56-9930-40f4-b1c0-86a0ba2cf530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:15.857804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671dd56-9930-40f4-b1c0-86a0ba2cf530 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:15.858121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671dd56-9930-40f4-b1c0-86a0ba2cf530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:15.858403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671dd56-9930-40f4-b1c0-86a0ba2cf530 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:15.859046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671dd56-9930-40f4-b1c0-86a0ba2cf530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:15.861085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671dd56-9930-40f4-b1c0-86a0ba2cf530 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:15.969776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 865/1706] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:15 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:15.974725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb2bb2-65c5-41f3-b597-952cfefa9a8b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:15.985744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb2bb2-65c5-41f3-b597-952cfefa9a8b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:15.991081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7d70f-e247-44d3-9ecc-1fb9f7c0bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:15.992085 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7d70f-e247-44d3-9ecc-1fb9f7c0bd05 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:15.992536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7d70f-e247-44d3-9ecc-1fb9f7c0bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:15.993219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7d70f-e247-44d3-9ecc-1fb9f7c0bd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:15.993951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7d70f-e247-44d3-9ecc-1fb9f7c0bd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:15.995678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7d70f-e247-44d3-9ecc-1fb9f7c0bd05 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:16.005720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 842/1707] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:16.010053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671dd56-9930-40f4-b1c0-86a0ba2cf530 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:16.021685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671dd56-9930-40f4-b1c0-86a0ba2cf530 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:16.026475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39431385-ead6-4663-b0ee-a11e9dc2b357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:16.027147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39431385-ead6-4663-b0ee-a11e9dc2b357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:16.027427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39431385-ead6-4663-b0ee-a11e9dc2b357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:16.027704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39431385-ead6-4663-b0ee-a11e9dc2b357 admin admin] PATH_INFO: `/v3/projects/9d6cfd5de8814a75bebb1d73ab9c6a60/users/b9a0ec148c2043c9ad8b88964948e693/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:16.043359 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39431385-ead6-4663-b0ee-a11e9dc2b357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6cfd5de8814a75bebb1d73ab9c6a60, user_id=b9a0ec148c2043c9ad8b88964948e693, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:16.046042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39431385-ead6-4663-b0ee-a11e9dc2b357 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:16.122391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 866/1708] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:16 2022] PUT /identity/v3/projects/9d6cfd5de8814a75bebb1d73ab9c6a60/users/b9a0ec148c2043c9ad8b88964948e693/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:16.126471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7d70f-e247-44d3-9ecc-1fb9f7c0bd05 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:16.137747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7d70f-e247-44d3-9ecc-1fb9f7c0bd05 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:16.199870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644f1df-c8d4-4457-966c-90e9989da039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:16.200482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644f1df-c8d4-4457-966c-90e9989da039 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:16.200757 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644f1df-c8d4-4457-966c-90e9989da039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:16.201026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644f1df-c8d4-4457-966c-90e9989da039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:16.201668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644f1df-c8d4-4457-966c-90e9989da039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:16.203296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644f1df-c8d4-4457-966c-90e9989da039 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:16.213868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 843/1709] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:16.217864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39431385-ead6-4663-b0ee-a11e9dc2b357 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:16.230930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39431385-ead6-4663-b0ee-a11e9dc2b357 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:16.235929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:16.236592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:16.236876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:16.241563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] PATH_INFO: `/v3/projects/9d6cfd5de8814a75bebb1d73ab9c6a60/users/b9a0ec148c2043c9ad8b88964948e693/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:16.241563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6cfd5de8814a75bebb1d73ab9c6a60, user_id=b9a0ec148c2043c9ad8b88964948e693, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:16.242762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:16.254132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 867/1710] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:16 2022] PUT /identity/v3/projects/9d6cfd5de8814a75bebb1d73ab9c6a60/users/b9a0ec148c2043c9ad8b88964948e693/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:16.557893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:16.615599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:17.537187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258ffe7-cf04-41f8-8f21-ca0a4dd0326c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:17.537476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258ffe7-cf04-41f8-8f21-ca0a4dd0326c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:17.537771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258ffe7-cf04-41f8-8f21-ca0a4dd0326c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:17.571952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f258ffe7-cf04-41f8-8f21-ca0a4dd0326c None None] MFA Rules not processed for user `b9a0ec148c2043c9ad8b88964948e693`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:17.573175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f258ffe7-cf04-41f8-8f21-ca0a4dd0326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:17.669108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 844/1711] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:17.683772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:17.683772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:17.701023 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:17.716362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:17.747117 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc084b5-1518-4456-8cbe-243042ef949c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:17.750709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30501d6f-034e-42a0-aa51-26fc9e981ebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:17.751310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30501d6f-034e-42a0-aa51-26fc9e981ebf service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:17.751588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30501d6f-034e-42a0-aa51-26fc9e981ebf service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:17.751853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30501d6f-034e-42a0-aa51-26fc9e981ebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:17.808273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30501d6f-034e-42a0-aa51-26fc9e981ebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:17.809802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30501d6f-034e-42a0-aa51-26fc9e981ebf service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:17.821024 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 868/1712] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:18.483970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:18.556081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258ffe7-cf04-41f8-8f21-ca0a4dd0326c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:18.557590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f258ffe7-cf04-41f8-8f21-ca0a4dd0326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:18.569607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258ffe7-cf04-41f8-8f21-ca0a4dd0326c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:18.573415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:18.576446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:18.576446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:18.576446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:18.576446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:18.638360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:18.639821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:18.650107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 845/1713] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:19.663823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30501d6f-034e-42a0-aa51-26fc9e981ebf service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:19.673224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30501d6f-034e-42a0-aa51-26fc9e981ebf service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:19.677627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d033eb-deff-4c6f-bbd9-45ecd2499389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:19.678271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d033eb-deff-4c6f-bbd9-45ecd2499389 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:19.678554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d033eb-deff-4c6f-bbd9-45ecd2499389 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:19.678830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d033eb-deff-4c6f-bbd9-45ecd2499389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:19.688147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d033eb-deff-4c6f-bbd9-45ecd2499389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:19.689632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d033eb-deff-4c6f-bbd9-45ecd2499389 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:19.700077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 869/1714] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:23.142038 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:23.143479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:23.256885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8278dd0-d489-40e5-a1d9-fb6b947881eb service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:23.261862 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:23.262538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:23.262816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:23.263085 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] PATH_INFO: `/v3/users/d64cc730513c46e6b0133b284dfdd0a9` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:23.264368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d64cc730513c46e6b0133b284dfdd0a9)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:23.266590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:23.457128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd64cc730513c46e6b0133b284dfdd0a9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:23.458325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:23.469797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd64cc730513c46e6b0133b284dfdd0a9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:23.567498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd64cc730513c46e6b0133b284dfdd0a9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:23.568066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd64cc730513c46e6b0133b284dfdd0a9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:23.575947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd64cc730513c46e6b0133b284dfdd0a9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:23.582950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 846/1715] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:23 2022] DELETE /identity/v3/users/d64cc730513c46e6b0133b284dfdd0a9 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:23.587405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fcd56-b985-4ca3-8336-99a4798d72c8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:23.587891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fcd56-b985-4ca3-8336-99a4798d72c8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:23.587980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fcd56-b985-4ca3-8336-99a4798d72c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:23.611678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-bd4fcd56-b985-4ca3-8336-99a4798d72c8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:23.611678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bd4fcd56-b985-4ca3-8336-99a4798d72c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:23.685391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 870/1716] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:23.696933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:23.707404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:23.764971 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944527a-500b-4a9d-8c9a-d9094b26bf1f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:23.768688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262b5a8-ab1c-4e4d-b9dd-c234137cf93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:23.769846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262b5a8-ab1c-4e4d-b9dd-c234137cf93a service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:23.770149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262b5a8-ab1c-4e4d-b9dd-c234137cf93a service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:23.770419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262b5a8-ab1c-4e4d-b9dd-c234137cf93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:23.779646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262b5a8-ab1c-4e4d-b9dd-c234137cf93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:23.781053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262b5a8-ab1c-4e4d-b9dd-c234137cf93a service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:23.790680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 847/1717] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:23.935180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb87e8-ea33-4d44-95ca-2df6a874f8f5 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:23.935374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb87e8-ea33-4d44-95ca-2df6a874f8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:23.935822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb87e8-ea33-4d44-95ca-2df6a874f8f5 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:23.937324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 871/1718] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:39:23.970637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925685fe-8940-4355-8ccb-64946b60995b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:23.971005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925685fe-8940-4355-8ccb-64946b60995b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:23.971210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925685fe-8940-4355-8ccb-64946b60995b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:24.000225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-925685fe-8940-4355-8ccb-64946b60995b None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:24.001588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-925685fe-8940-4355-8ccb-64946b60995b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:24.063071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb87e8-ea33-4d44-95ca-2df6a874f8f5 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:24.064827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-dfbb87e8-ea33-4d44-95ca-2df6a874f8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:24.090401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb87e8-ea33-4d44-95ca-2df6a874f8f5 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:24.099260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:24.100024 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:24.100585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:24.100930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] PATH_INFO: `/v3/users/df838ad816744832baa283dabe540394` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:24.102214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=df838ad816744832baa283dabe540394)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:24.104209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:24.113970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 848/1719] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:24.122253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71179d26-f8e8-45f9-9d6e-9cf86efa5e53 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:24.122545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71179d26-f8e8-45f9-9d6e-9cf86efa5e53 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:24.122820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71179d26-f8e8-45f9-9d6e-9cf86efa5e53 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:24.124304 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 849/1720] 10.222.0.44 () {54 vars in 961 bytes} [Wed Oct 12 12:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:39:24.129769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f836f-fce7-409d-a3b3-bef2936c9ed8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:24.130086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f836f-fce7-409d-a3b3-bef2936c9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:24.130359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f836f-fce7-409d-a3b3-bef2936c9ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:24.152359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-563f836f-fce7-409d-a3b3-bef2936c9ed8 None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:24.153578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-563f836f-fce7-409d-a3b3-bef2936c9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:24.166088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 850/1721] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Oct 12 12:39:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:24.173471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f836f-fce7-409d-a3b3-bef2936c9ed8 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:24.188325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f836f-fce7-409d-a3b3-bef2936c9ed8 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:24.195235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:24.197773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:24.197773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:24.197773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:24.205459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:24.207279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:24.217293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 851/1722] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:24.235243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:24.241162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:24.263859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24bc7ba-83fc-4ab2-99f1-b71dc1f443eb service placement] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:24.276124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:24.278269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:24.278269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:24.278269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] PATH_INFO: `/v3/projects/b2ca25970dd640e4939d88a06a74b795` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:24.281533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ca25970dd640e4939d88a06a74b795)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:24.284272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:24.316548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df838ad816744832baa283dabe540394'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.330629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df838ad816744832baa283dabe540394'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.331478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df838ad816744832baa283dabe540394'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.333229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:24.395861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df838ad816744832baa283dabe540394'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.403717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df838ad816744832baa283dabe540394'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.409446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:24.411015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 872/1723] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:24 2022] DELETE /identity/v3/users/df838ad816744832baa283dabe540394 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:24.424512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ca25970dd640e4939d88a06a74b795', 'request_id': 'req-0de2337f-514d-4468-b6a5-0c139498b209'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.433526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ca25970dd640e4939d88a06a74b795', 'request_id': 'req-0de2337f-514d-4468-b6a5-0c139498b209'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.440797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ca25970dd640e4939d88a06a74b795', 'request_id': 'req-0de2337f-514d-4468-b6a5-0c139498b209'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.523424 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] The token cache is being invalidate because project b2ca25970dd640e4939d88a06a74b795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:24.523720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.524458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.525878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 852/1724] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:24 2022] DELETE /identity/v3/projects/b2ca25970dd640e4939d88a06a74b795 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:24.529887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:24.531455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:24.614389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03004a3-877a-4b79-9c73-421a4f661c4c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:24.619566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:24.620196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:24.620476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:24.620750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] PATH_INFO: `/v3/users/2befdeb24cea4c82ba66ad889bbecd67` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:24.621868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2befdeb24cea4c82ba66ad889bbecd67)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:24.623959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:24.707385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:24.709003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:24.794861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2befdeb24cea4c82ba66ad889bbecd67'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.810581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2befdeb24cea4c82ba66ad889bbecd67'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.811287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2befdeb24cea4c82ba66ad889bbecd67'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.826096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2337f-514d-4468-b6a5-0c139498b209 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:24.831083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:24.831748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:24.832035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:24.832313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] PATH_INFO: `/v3/projects/677a44a0d7bb436f83c23a8e74cffa60` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:24.833967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=677a44a0d7bb436f83c23a8e74cffa60)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:24.835803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:24.895031 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2befdeb24cea4c82ba66ad889bbecd67'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.902604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4b61c1a6-7e94-4b05-b067-44da7412b9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2befdeb24cea4c82ba66ad889bbecd67'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.908594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 873/1725] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:24 2022] DELETE /identity/v3/users/2befdeb24cea4c82ba66ad889bbecd67 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:24.940156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '677a44a0d7bb436f83c23a8e74cffa60', 'request_id': 'req-396d1969-b1ed-4c82-adc8-a060dc6ab316'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.946353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '677a44a0d7bb436f83c23a8e74cffa60', 'request_id': 'req-396d1969-b1ed-4c82-adc8-a060dc6ab316'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:24.958826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '677a44a0d7bb436f83c23a8e74cffa60', 'request_id': 'req-396d1969-b1ed-4c82-adc8-a060dc6ab316'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:25.037979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] The token cache is being invalidate because project 677a44a0d7bb436f83c23a8e74cffa60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:25.038275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:25.038923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-396d1969-b1ed-4c82-adc8-a060dc6ab316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:25.040464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 853/1726] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:24 2022] DELETE /identity/v3/projects/677a44a0d7bb436f83c23a8e74cffa60 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:25.045604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84420761-a9a4-4c55-9515-e8a295abc221 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.045909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84420761-a9a4-4c55-9515-e8a295abc221 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.046213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84420761-a9a4-4c55-9515-e8a295abc221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.067607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-84420761-a9a4-4c55-9515-e8a295abc221 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:25.070230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-84420761-a9a4-4c55-9515-e8a295abc221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:25.139445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 874/1727] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:25.151605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8638-891b-423d-93ed-0f7ede9018ed None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.151605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8638-891b-423d-93ed-0f7ede9018ed None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.151605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8638-891b-423d-93ed-0f7ede9018ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.171963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-704f8638-891b-423d-93ed-0f7ede9018ed None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:25.174044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-704f8638-891b-423d-93ed-0f7ede9018ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:25.184864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 854/1728] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:25.189332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84420761-a9a4-4c55-9515-e8a295abc221 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.194288 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:25.199840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84420761-a9a4-4c55-9515-e8a295abc221 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.218291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.218291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.218291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.218291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.218817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:25.218817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.218817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.229987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 875/1729] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:25.230443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f8638-891b-423d-93ed-0f7ede9018ed None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.240170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f8638-891b-423d-93ed-0f7ede9018ed None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.246132 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.246740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.249119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.249119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.249119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.251287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.362703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 855/1730] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:25 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:25.363922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.375280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc4465-a4c6-4164-b00b-ac346deb188e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.401024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.402565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:25.446667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9dd5-6438-4d58-8a04-b7ea5ad2e127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.449635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9dd5-6438-4d58-8a04-b7ea5ad2e127 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.449635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9dd5-6438-4d58-8a04-b7ea5ad2e127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.449635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9dd5-6438-4d58-8a04-b7ea5ad2e127 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.449635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a9dd5-6438-4d58-8a04-b7ea5ad2e127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.450028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:25.450454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a9dd5-6438-4d58-8a04-b7ea5ad2e127 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.485642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06bd46-841a-4025-9d35-ee9b5068bd17 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.491057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.491950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.492232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.492505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] PATH_INFO: `/v3/projects/9eb6c88e7c924b4f82244cb74465475d` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.493731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb6c88e7c924b4f82244cb74465475d)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.495998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.522018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 876/1731] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:25 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:25.523345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9dd5-6438-4d58-8a04-b7ea5ad2e127 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.533456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9dd5-6438-4d58-8a04-b7ea5ad2e127 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.538868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72292bcb-75b2-48c9-8738-b4b167d0acca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.539495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72292bcb-75b2-48c9-8738-b4b167d0acca admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.539800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72292bcb-75b2-48c9-8738-b4b167d0acca admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.540180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72292bcb-75b2-48c9-8738-b4b167d0acca admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.540824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72292bcb-75b2-48c9-8738-b4b167d0acca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.542442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72292bcb-75b2-48c9-8738-b4b167d0acca admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.554180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 877/1732] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:25.558129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72292bcb-75b2-48c9-8738-b4b167d0acca admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.568919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72292bcb-75b2-48c9-8738-b4b167d0acca admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.573802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb8927-e139-41f2-96aa-d045112a5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.574462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb8927-e139-41f2-96aa-d045112a5a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.574743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb8927-e139-41f2-96aa-d045112a5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.575022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb8927-e139-41f2-96aa-d045112a5a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.575674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb8927-e139-41f2-96aa-d045112a5a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.577258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb8927-e139-41f2-96aa-d045112a5a45 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.588451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 878/1733] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:25.592554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb8927-e139-41f2-96aa-d045112a5a45 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.602490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb8927-e139-41f2-96aa-d045112a5a45 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.607307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b639c06-9dfd-4590-99b3-1bdcfefe8995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.608016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639c06-9dfd-4590-99b3-1bdcfefe8995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.608295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639c06-9dfd-4590-99b3-1bdcfefe8995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.608611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639c06-9dfd-4590-99b3-1bdcfefe8995 admin admin] PATH_INFO: `/v3/projects/f3276356aa124d428c2bf6bbf2fc6dac/users/f2d21244b4ae49e8a535de8d4312e09d/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.625176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b639c06-9dfd-4590-99b3-1bdcfefe8995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3276356aa124d428c2bf6bbf2fc6dac, user_id=f2d21244b4ae49e8a535de8d4312e09d, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.628837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b639c06-9dfd-4590-99b3-1bdcfefe8995 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.644460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb6c88e7c924b4f82244cb74465475d', 'request_id': 'req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:25.649961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb6c88e7c924b4f82244cb74465475d', 'request_id': 'req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:25.656141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb6c88e7c924b4f82244cb74465475d', 'request_id': 'req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:25.706561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 879/1734] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:25 2022] PUT /identity/v3/projects/f3276356aa124d428c2bf6bbf2fc6dac/users/f2d21244b4ae49e8a535de8d4312e09d/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:25.716623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0997071-1ab7-49e3-b075-00bb6d5da2f4 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.716912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0997071-1ab7-49e3-b075-00bb6d5da2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.717188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0997071-1ab7-49e3-b075-00bb6d5da2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.728970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] The token cache is being invalidate because project 9eb6c88e7c924b4f82244cb74465475d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:25.729258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:25.730014 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:25.731620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 856/1735] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:25 2022] DELETE /identity/v3/projects/9eb6c88e7c924b4f82244cb74465475d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:25.735433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.736710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:25.746686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e0997071-1ab7-49e3-b075-00bb6d5da2f4 None None] MFA Rules not processed for user `f2d21244b4ae49e8a535de8d4312e09d`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:25.747916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e0997071-1ab7-49e3-b075-00bb6d5da2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:25.806090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60db2f-5d07-4874-8c18-8a5a66fc27c6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.815345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 880/1736] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:25.829318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0997071-1ab7-49e3-b075-00bb6d5da2f4 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:25.831444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e0997071-1ab7-49e3-b075-00bb6d5da2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:25.866315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:25.881920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.883883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.883883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.883883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] PATH_INFO: `/v3/users/7d5b86d11ed340a8b06f89b5b44326c6` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.884333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d5b86d11ed340a8b06f89b5b44326c6)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.886291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.892104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:25.916178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0997071-1ab7-49e3-b075-00bb6d5da2f4 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:25.921592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa617c2-1564-4de3-84da-61bb21f0783b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:25.922224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa617c2-1564-4de3-84da-61bb21f0783b service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:25.922504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa617c2-1564-4de3-84da-61bb21f0783b service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:25.922774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa617c2-1564-4de3-84da-61bb21f0783b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:25.931202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa617c2-1564-4de3-84da-61bb21f0783b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:25.932767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa617c2-1564-4de3-84da-61bb21f0783b service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:25.942224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 881/1737] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:26.079835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d5b86d11ed340a8b06f89b5b44326c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:26.092596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d5b86d11ed340a8b06f89b5b44326c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:26.238892 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d5b86d11ed340a8b06f89b5b44326c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:26.239561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d5b86d11ed340a8b06f89b5b44326c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:26.247484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-9d5783e8-e3eb-486d-8d8e-03ee5833b574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d5b86d11ed340a8b06f89b5b44326c6'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:26.253619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 857/1738] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:25 2022] DELETE /identity/v3/users/7d5b86d11ed340a8b06f89b5b44326c6 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:26.715963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa617c2-1564-4de3-84da-61bb21f0783b service nova] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:26.747391 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa617c2-1564-4de3-84da-61bb21f0783b service nova] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:26.849812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:26.850501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:26.850787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:26.851088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] PATH_INFO: `/v3/projects/c4c203138528419ba9515e3f48095ea5` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:26.852246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4c203138528419ba9515e3f48095ea5)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:26.854283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:27.016353 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4c203138528419ba9515e3f48095ea5', 'request_id': 'req-762f3581-f39b-4a8d-9b7a-6a897e3acb51'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:27.022695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4c203138528419ba9515e3f48095ea5', 'request_id': 'req-762f3581-f39b-4a8d-9b7a-6a897e3acb51'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:27.029490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4c203138528419ba9515e3f48095ea5', 'request_id': 'req-762f3581-f39b-4a8d-9b7a-6a897e3acb51'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:27.111440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] The token cache is being invalidate because project c4c203138528419ba9515e3f48095ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:27.111730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:27.112351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-762f3581-f39b-4a8d-9b7a-6a897e3acb51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:27.114392 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 882/1739] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:26 2022] DELETE /identity/v3/projects/c4c203138528419ba9515e3f48095ea5 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:27.120636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfa2a3-cb7a-4aed-9a5d-0bafca8d4efd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.120927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfa2a3-cb7a-4aed-9a5d-0bafca8d4efd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.121202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfa2a3-cb7a-4aed-9a5d-0bafca8d4efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.142439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b2cfa2a3-cb7a-4aed-9a5d-0bafca8d4efd None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:27.143695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b2cfa2a3-cb7a-4aed-9a5d-0bafca8d4efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:27.202065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 858/1740] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:27.210911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472093f-8e53-4f1f-84a0-c37bd546415c None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.211189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472093f-8e53-4f1f-84a0-c37bd546415c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.211461 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472093f-8e53-4f1f-84a0-c37bd546415c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.230959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c472093f-8e53-4f1f-84a0-c37bd546415c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:27.232141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c472093f-8e53-4f1f-84a0-c37bd546415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:27.240329 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 883/1741] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:27.246059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfa2a3-cb7a-4aed-9a5d-0bafca8d4efd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:27.255919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfa2a3-cb7a-4aed-9a5d-0bafca8d4efd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:27.260480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:27.261100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.261382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.261656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.262340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:27.262645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:27.264224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:27.276267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 859/1742] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:27.280446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472093f-8e53-4f1f-84a0-c37bd546415c None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:27.290209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472093f-8e53-4f1f-84a0-c37bd546415c None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:27.294815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2e129-7167-479d-bd0b-2c7917b32f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:27.295401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2e129-7167-479d-bd0b-2c7917b32f52 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.295677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2e129-7167-479d-bd0b-2c7917b32f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.295943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2e129-7167-479d-bd0b-2c7917b32f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.296572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2e129-7167-479d-bd0b-2c7917b32f52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:27.298493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2e129-7167-479d-bd0b-2c7917b32f52 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:27.376227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 884/1743] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:27 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:27.380392 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:27.390664 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba64b-be0a-47c3-bb67-81243937354a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:27.448758 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c9b2-c4e1-4c9b-bcaa-eb479f7208c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:27.449351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c9b2-c4e1-4c9b-bcaa-eb479f7208c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.449632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c9b2-c4e1-4c9b-bcaa-eb479f7208c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.449905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915c9b2-c4e1-4c9b-bcaa-eb479f7208c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.450563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915c9b2-c4e1-4c9b-bcaa-eb479f7208c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:27.452465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915c9b2-c4e1-4c9b-bcaa-eb479f7208c5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:27.546583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 860/1744] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:27 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:27.550724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2e129-7167-479d-bd0b-2c7917b32f52 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:27.561325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2e129-7167-479d-bd0b-2c7917b32f52 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:27.566497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2b7e2-c404-4694-8180-12eb9f0af085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:27.567106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2b7e2-c404-4694-8180-12eb9f0af085 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.567426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2b7e2-c404-4694-8180-12eb9f0af085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.567701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2b7e2-c404-4694-8180-12eb9f0af085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.568336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2b7e2-c404-4694-8180-12eb9f0af085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:27.570019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2b7e2-c404-4694-8180-12eb9f0af085 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:27.572611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:27.580776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 885/1745] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:27.585045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c9b2-c4e1-4c9b-bcaa-eb479f7208c5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:27.590560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:27.596025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915c9b2-c4e1-4c9b-bcaa-eb479f7208c5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:27.600588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba854e52-9e6c-4fcd-ba20-33723593bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:27.601274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba854e52-9e6c-4fcd-ba20-33723593bead admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.601478 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba854e52-9e6c-4fcd-ba20-33723593bead admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.601751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba854e52-9e6c-4fcd-ba20-33723593bead admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.602421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba854e52-9e6c-4fcd-ba20-33723593bead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:27.604033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba854e52-9e6c-4fcd-ba20-33723593bead admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:27.614879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 861/1746] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:27.619277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2b7e2-c404-4694-8180-12eb9f0af085 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:27.630375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2b7e2-c404-4694-8180-12eb9f0af085 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:27.635095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae01e9-0478-4960-bf03-0e2bcd08d9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:27.635739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae01e9-0478-4960-bf03-0e2bcd08d9dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.636113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae01e9-0478-4960-bf03-0e2bcd08d9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.636414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae01e9-0478-4960-bf03-0e2bcd08d9dd admin admin] PATH_INFO: `/v3/projects/2a54ec482a8a40e697a172173b246227/users/05f13142840d4b72a2a1eab04e94ee1f/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.652591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae01e9-0478-4960-bf03-0e2bcd08d9dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a54ec482a8a40e697a172173b246227, user_id=05f13142840d4b72a2a1eab04e94ee1f, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:27.655318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae01e9-0478-4960-bf03-0e2bcd08d9dd admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:27.746779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 886/1747] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:27 2022] PUT /identity/v3/projects/2a54ec482a8a40e697a172173b246227/users/05f13142840d4b72a2a1eab04e94ee1f/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:27.758385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33517d6-6c07-48ec-b227-d00b6f33ee36 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.758669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33517d6-6c07-48ec-b227-d00b6f33ee36 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.758949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33517d6-6c07-48ec-b227-d00b6f33ee36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.786896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d33517d6-6c07-48ec-b227-d00b6f33ee36 None None] MFA Rules not processed for user `05f13142840d4b72a2a1eab04e94ee1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:27.788098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d33517d6-6c07-48ec-b227-d00b6f33ee36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:27.868549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 862/1748] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:27.873110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae01e9-0478-4960-bf03-0e2bcd08d9dd admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:27.953826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae01e9-0478-4960-bf03-0e2bcd08d9dd admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:27.958842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca14b-d9d2-4c38-bc47-dca3d21fdf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:27.959602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca14b-d9d2-4c38-bc47-dca3d21fdf6a admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:27.959883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca14b-d9d2-4c38-bc47-dca3d21fdf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:27.960177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca14b-d9d2-4c38-bc47-dca3d21fdf6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:27.960823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ca14b-d9d2-4c38-bc47-dca3d21fdf6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:27.962930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ca14b-d9d2-4c38-bc47-dca3d21fdf6a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:28.075247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 887/1749] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:27 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:28.079378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33517d6-6c07-48ec-b227-d00b6f33ee36 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:28.090606 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33517d6-6c07-48ec-b227-d00b6f33ee36 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:28.152101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd753a-5b4e-45ed-a600-e97bd0e50588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:28.152796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd753a-5b4e-45ed-a600-e97bd0e50588 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:28.153155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd753a-5b4e-45ed-a600-e97bd0e50588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:28.153523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd753a-5b4e-45ed-a600-e97bd0e50588 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:28.154326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd753a-5b4e-45ed-a600-e97bd0e50588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:28.156438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd753a-5b4e-45ed-a600-e97bd0e50588 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:28.294110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 863/1750] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:28 2022] POST /identity/v3/users => generated 297 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:28.298093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca14b-d9d2-4c38-bc47-dca3d21fdf6a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:28.309348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca14b-d9d2-4c38-bc47-dca3d21fdf6a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:28.315345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb6d42-8986-48f8-a813-7a8c093c74c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:28.315973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb6d42-8986-48f8-a813-7a8c093c74c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:28.316251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb6d42-8986-48f8-a813-7a8c093c74c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:28.316523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb6d42-8986-48f8-a813-7a8c093c74c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:28.317167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb6d42-8986-48f8-a813-7a8c093c74c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:28.318784 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb6d42-8986-48f8-a813-7a8c093c74c7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:28.329624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 888/1751] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:28.333866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd753a-5b4e-45ed-a600-e97bd0e50588 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:28.344075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd753a-5b4e-45ed-a600-e97bd0e50588 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:28.349308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33cb60-cfca-4ca5-9136-8b1646915d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:28.349956 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33cb60-cfca-4ca5-9136-8b1646915d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:28.350318 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33cb60-cfca-4ca5-9136-8b1646915d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:28.350733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33cb60-cfca-4ca5-9136-8b1646915d27 admin admin] PATH_INFO: `/v3/projects/9aa3b1de858f4c01a6775ce850f8f825/users/40fdc87939864333aa92d6809b38f61c/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:28.365986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33cb60-cfca-4ca5-9136-8b1646915d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa3b1de858f4c01a6775ce850f8f825, user_id=40fdc87939864333aa92d6809b38f61c, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:28.369101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33cb60-cfca-4ca5-9136-8b1646915d27 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:28.466540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 864/1752] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:28 2022] PUT /identity/v3/projects/9aa3b1de858f4c01a6775ce850f8f825/users/40fdc87939864333aa92d6809b38f61c/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:28.470726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb6d42-8986-48f8-a813-7a8c093c74c7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:28.481478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb6d42-8986-48f8-a813-7a8c093c74c7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:28.543820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e5e06-92a8-4d66-aab8-e9191a342f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:28.544439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e5e06-92a8-4d66-aab8-e9191a342f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:28.544780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e5e06-92a8-4d66-aab8-e9191a342f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:28.544991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e5e06-92a8-4d66-aab8-e9191a342f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:28.545634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e5e06-92a8-4d66-aab8-e9191a342f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:28.547304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e5e06-92a8-4d66-aab8-e9191a342f8c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:28.558453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 889/1753] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:28.566746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33cb60-cfca-4ca5-9136-8b1646915d27 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:28.577360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33cb60-cfca-4ca5-9136-8b1646915d27 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:28.582246 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:28.583056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:28.583362 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:28.583636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] PATH_INFO: `/v3/projects/9aa3b1de858f4c01a6775ce850f8f825/users/40fdc87939864333aa92d6809b38f61c/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:28.585644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa3b1de858f4c01a6775ce850f8f825, user_id=40fdc87939864333aa92d6809b38f61c, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:28.588503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:28.600051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 865/1754] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:28 2022] PUT /identity/v3/projects/9aa3b1de858f4c01a6775ce850f8f825/users/40fdc87939864333aa92d6809b38f61c/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:28.610090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e811-f786-4a5d-9b9d-831a26e25635 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:28.610373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e811-f786-4a5d-9b9d-831a26e25635 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:28.610644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e811-f786-4a5d-9b9d-831a26e25635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:28.642767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-3121e811-f786-4a5d-9b9d-831a26e25635 None None] MFA Rules not processed for user `40fdc87939864333aa92d6809b38f61c`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:28.644511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3121e811-f786-4a5d-9b9d-831a26e25635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:28.679426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:28.719104 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 890/1755] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:28.731040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:28.732924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:28.769692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:28.793820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8fa02-8513-4f1b-bbb2-60c552a36523 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:28.797620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0afdf-ee4a-4bce-9d4c-374a0bb3633b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:28.799209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0afdf-ee4a-4bce-9d4c-374a0bb3633b service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:28.799209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0afdf-ee4a-4bce-9d4c-374a0bb3633b service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:28.799209 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0afdf-ee4a-4bce-9d4c-374a0bb3633b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:28.807729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0afdf-ee4a-4bce-9d4c-374a0bb3633b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:28.809141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0afdf-ee4a-4bce-9d4c-374a0bb3633b service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:28.819541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 866/1756] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:29.332804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e811-f786-4a5d-9b9d-831a26e25635 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:29.332804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3121e811-f786-4a5d-9b9d-831a26e25635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:29.353358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0afdf-ee4a-4bce-9d4c-374a0bb3633b service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:29.400282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e811-f786-4a5d-9b9d-831a26e25635 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:29.404835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:29.405459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:29.405744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:29.406040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] PATH_INFO: `/v3/users/f2d21244b4ae49e8a535de8d4312e09d` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:29.407151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d21244b4ae49e8a535de8d4312e09d)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:29.409050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:29.432756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0afdf-ee4a-4bce-9d4c-374a0bb3633b service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:29.437959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:29.438885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:29.438885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:29.439162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] PATH_INFO: `/v3/users/05f13142840d4b72a2a1eab04e94ee1f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:29.440436 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05f13142840d4b72a2a1eab04e94ee1f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:29.443068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:29.624137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d21244b4ae49e8a535de8d4312e09d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.637319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d21244b4ae49e8a535de8d4312e09d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.638083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d21244b4ae49e8a535de8d4312e09d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.685893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05f13142840d4b72a2a1eab04e94ee1f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.703782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05f13142840d4b72a2a1eab04e94ee1f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.706579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d21244b4ae49e8a535de8d4312e09d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.716107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ef37e34-074b-4409-877a-7ca59e6675c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d21244b4ae49e8a535de8d4312e09d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.722871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 891/1757] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:29 2022] DELETE /identity/v3/users/f2d21244b4ae49e8a535de8d4312e09d => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:29.727480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf54389-601c-4e47-8bc4-0b66ddd4d520 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:29.727863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf54389-601c-4e47-8bc4-0b66ddd4d520 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:29.728159 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf54389-601c-4e47-8bc4-0b66ddd4d520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:29.747975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-bbf54389-601c-4e47-8bc4-0b66ddd4d520 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:29.749205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bbf54389-601c-4e47-8bc4-0b66ddd4d520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:29.793249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:29.809179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 892/1758] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:29.809401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05f13142840d4b72a2a1eab04e94ee1f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.809876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05f13142840d4b72a2a1eab04e94ee1f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.817998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c9008485-3cd1-42bd-9b89-ed7e25f00289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05f13142840d4b72a2a1eab04e94ee1f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:29.818880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf54389-601c-4e47-8bc4-0b66ddd4d520 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:29.820279 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bbf54389-601c-4e47-8bc4-0b66ddd4d520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:29.824049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 867/1759] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:29 2022] DELETE /identity/v3/users/05f13142840d4b72a2a1eab04e94ee1f => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:29.828273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b20a4-7bf9-4c41-8725-7e2fe700bd0a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:29.828565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b20a4-7bf9-4c41-8725-7e2fe700bd0a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:29.828866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b20a4-7bf9-4c41-8725-7e2fe700bd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:29.864929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8d0b20a4-7bf9-4c41-8725-7e2fe700bd0a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:29.867963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8d0b20a4-7bf9-4c41-8725-7e2fe700bd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:29.910686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf54389-601c-4e47-8bc4-0b66ddd4d520 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:29.936803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:29.957394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 868/1760] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:29.963756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:29.964033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b20a4-7bf9-4c41-8725-7e2fe700bd0a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:29.964595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:29.964930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:29.965255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:29.966747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8d0b20a4-7bf9-4c41-8725-7e2fe700bd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:29.967262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:29.978099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b20a4-7bf9-4c41-8725-7e2fe700bd0a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:29.980166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:29.981743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:29.982632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:29.983355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:29.983639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:29.984004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:29.994988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:29.995902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 893/1761] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:29.996571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:30.006503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 869/1762] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:30.481045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:30.481841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:30.482820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:30.484018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:30.498092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18168b48-1911-40e0-a077-b90f9a7801f4 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:30.499914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab49844-8d64-4e82-ae58-8eeddbabdccd service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:30.510975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:30.512485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:30.512604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:30.512876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:30.513161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] PATH_INFO: `/v3/projects/f3276356aa124d428c2bf6bbf2fc6dac` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:30.513257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:30.513583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:30.514142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] PATH_INFO: `/v3/projects/2a54ec482a8a40e697a172173b246227` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:30.514624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3276356aa124d428c2bf6bbf2fc6dac)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:30.515655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a54ec482a8a40e697a172173b246227)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:30.516873 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:30.518135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:30.622581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3276356aa124d428c2bf6bbf2fc6dac', 'request_id': 'req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.629369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3276356aa124d428c2bf6bbf2fc6dac', 'request_id': 'req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.644647 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a54ec482a8a40e697a172173b246227', 'request_id': 'req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.645691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3276356aa124d428c2bf6bbf2fc6dac', 'request_id': 'req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.655542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a54ec482a8a40e697a172173b246227', 'request_id': 'req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.664439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a54ec482a8a40e697a172173b246227', 'request_id': 'req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.711982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] The token cache is being invalidate because project f3276356aa124d428c2bf6bbf2fc6dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:30.712272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.712925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7dc036f4-a6c0-4dba-8e3d-4626b7a39eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.714529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 894/1763] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:30 2022] DELETE /identity/v3/projects/f3276356aa124d428c2bf6bbf2fc6dac => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:30.719075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eece8c7-3b00-4dd8-86ac-2386c3ffee1a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:30.719355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eece8c7-3b00-4dd8-86ac-2386c3ffee1a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:30.719627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eece8c7-3b00-4dd8-86ac-2386c3ffee1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:30.738818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-3eece8c7-3b00-4dd8-86ac-2386c3ffee1a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:30.740005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3eece8c7-3b00-4dd8-86ac-2386c3ffee1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:30.767644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:30.801686 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] The token cache is being invalidate because project 2a54ec482a8a40e697a172173b246227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:30.802021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.802606 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:30.804184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 870/1764] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:30 2022] DELETE /identity/v3/projects/2a54ec482a8a40e697a172173b246227 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:30.806652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 895/1765] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:30.808075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:30.809384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:30.818226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6b3bc-cc8c-4ea1-84e6-049d83ecf0f5 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:30.818608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6b3bc-cc8c-4ea1-84e6-049d83ecf0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:30.818918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6b3bc-cc8c-4ea1-84e6-049d83ecf0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:30.826359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311b59-4e2c-4d30-8e43-2ec53eb4270c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:30.831065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:30.832522 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:30.832801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:30.833073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] PATH_INFO: `/v3/users/40fdc87939864333aa92d6809b38f61c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:30.834266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40fdc87939864333aa92d6809b38f61c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:30.836200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:30.840658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7bd6b3bc-cc8c-4ea1-84e6-049d83ecf0f5 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:30.843023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7bd6b3bc-cc8c-4ea1-84e6-049d83ecf0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:30.851986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 896/1766] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:30.856227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6b3bc-cc8c-4ea1-84e6-049d83ecf0f5 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:30.866154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6b3bc-cc8c-4ea1-84e6-049d83ecf0f5 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:30.871109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:30.871899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:30.872469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:30.872969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:30.873674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:30.874460 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:30.877371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:30.888463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 897/1767] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:30.892645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:30.906840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539c897-9a51-49f4-9401-f70efa1f6431 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:30.908348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cccfd-ee0e-4c3c-b9c5-b8dea7cc0a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:30.908990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cccfd-ee0e-4c3c-b9c5-b8dea7cc0a1c admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:30.909272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cccfd-ee0e-4c3c-b9c5-b8dea7cc0a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:30.909546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6cccfd-ee0e-4c3c-b9c5-b8dea7cc0a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:30.910290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cccfd-ee0e-4c3c-b9c5-b8dea7cc0a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:30.912272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6cccfd-ee0e-4c3c-b9c5-b8dea7cc0a1c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:31.023414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40fdc87939864333aa92d6809b38f61c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.032382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 898/1768] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:30 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:31.033470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cccfd-ee0e-4c3c-b9c5-b8dea7cc0a1c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:31.037387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40fdc87939864333aa92d6809b38f61c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.101079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40fdc87939864333aa92d6809b38f61c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.101789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40fdc87939864333aa92d6809b38f61c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.102694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6cccfd-ee0e-4c3c-b9c5-b8dea7cc0a1c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:31.107769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527968dd-d974-4a02-908b-89fdb27daddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:31.108335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527968dd-d974-4a02-908b-89fdb27daddb admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.113463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527968dd-d974-4a02-908b-89fdb27daddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.113463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527968dd-d974-4a02-908b-89fdb27daddb admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.113463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527968dd-d974-4a02-908b-89fdb27daddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:31.113728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40fdc87939864333aa92d6809b38f61c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.113830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527968dd-d974-4a02-908b-89fdb27daddb admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:31.116900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 871/1769] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:30 2022] DELETE /identity/v3/users/40fdc87939864333aa92d6809b38f61c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:31.227672 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 899/1770] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:31 2022] POST /identity/v3/users => generated 303 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:31.231857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:31.233437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:31.277859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:31.301780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fc654-bbb2-487f-92f4-74f5b19cb8c1 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:31.306888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4a449-c4eb-448a-a30a-6bafaec19395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:31.307503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4a449-c4eb-448a-a30a-6bafaec19395 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.307993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4a449-c4eb-448a-a30a-6bafaec19395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.308120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4a449-c4eb-448a-a30a-6bafaec19395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.308986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4a449-c4eb-448a-a30a-6bafaec19395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:31.310654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4a449-c4eb-448a-a30a-6bafaec19395 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:31.337287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 872/1771] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:31.343038 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527968dd-d974-4a02-908b-89fdb27daddb admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:31.356602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527968dd-d974-4a02-908b-89fdb27daddb admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:31.365578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc289a-7db6-429b-a746-235cdb3d9101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:31.366229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc289a-7db6-429b-a746-235cdb3d9101 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.366512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc289a-7db6-429b-a746-235cdb3d9101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.367055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc289a-7db6-429b-a746-235cdb3d9101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.367830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc289a-7db6-429b-a746-235cdb3d9101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:31.369637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc289a-7db6-429b-a746-235cdb3d9101 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:31.380915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 900/1772] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:31.384691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4a449-c4eb-448a-a30a-6bafaec19395 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:31.395173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4a449-c4eb-448a-a30a-6bafaec19395 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:31.400167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:31.400949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.401306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.401658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] PATH_INFO: `/v3/projects/c87dc51bd48d4f72a26ae2fd676ff73f/users/73f7c5ee59874429a4ecbee351702aa0/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.417972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87dc51bd48d4f72a26ae2fd676ff73f, user_id=73f7c5ee59874429a4ecbee351702aa0, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:31.421984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:31.510948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 873/1773] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:31 2022] PUT /identity/v3/projects/c87dc51bd48d4f72a26ae2fd676ff73f/users/73f7c5ee59874429a4ecbee351702aa0/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:31.516122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a6bca-1623-44b0-83e0-285436c0cffc None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.516561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a6bca-1623-44b0-83e0-285436c0cffc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.516679 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a6bca-1623-44b0-83e0-285436c0cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.539310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-975a6bca-1623-44b0-83e0-285436c0cffc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:31.540518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-975a6bca-1623-44b0-83e0-285436c0cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:31.581553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:31.585147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:31.630859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 901/1774] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:31.641449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a6bca-1623-44b0-83e0-285436c0cffc None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:31.643012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-975a6bca-1623-44b0-83e0-285436c0cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:31.658006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:31.689121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560e555-0737-4722-9f55-2a61ec8b96d3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:31.693763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:31.694441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.694719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.694990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] PATH_INFO: `/v3/projects/9aa3b1de858f4c01a6775ce850f8f825` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.696239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa3b1de858f4c01a6775ce850f8f825)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:31.698523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:31.723041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a6bca-1623-44b0-83e0-285436c0cffc None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:31.728247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338df16-27f0-4bc1-90de-076cffd1806e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:31.728933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338df16-27f0-4bc1-90de-076cffd1806e service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.729224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338df16-27f0-4bc1-90de-076cffd1806e service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.729483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338df16-27f0-4bc1-90de-076cffd1806e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.743040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338df16-27f0-4bc1-90de-076cffd1806e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:31.744609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338df16-27f0-4bc1-90de-076cffd1806e service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:31.760290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 902/1775] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:31.829242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa3b1de858f4c01a6775ce850f8f825', 'request_id': 'req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.836202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa3b1de858f4c01a6775ce850f8f825', 'request_id': 'req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.843068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa3b1de858f4c01a6775ce850f8f825', 'request_id': 'req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.936627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] The token cache is being invalidate because project 9aa3b1de858f4c01a6775ce850f8f825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:31.936929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.937615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-827ad8a6-6062-4bf2-9dbe-0a34db6e2a07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:31.939625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 874/1776] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:31 2022] DELETE /identity/v3/projects/9aa3b1de858f4c01a6775ce850f8f825 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:31.944914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4787ace-cc32-40ae-9713-cd69e4adca7e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.945322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4787ace-cc32-40ae-9713-cd69e4adca7e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.946181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4787ace-cc32-40ae-9713-cd69e4adca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.967880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c4787ace-cc32-40ae-9713-cd69e4adca7e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:31.969492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c4787ace-cc32-40ae-9713-cd69e4adca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:32.031094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 903/1777] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:32.040254 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7b6b-32c8-489a-a020-1747c50fe84f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.040521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7b6b-32c8-489a-a020-1747c50fe84f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.040795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7b6b-32c8-489a-a020-1747c50fe84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.061124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bb7f7b6b-32c8-489a-a020-1747c50fe84f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:32.062365 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bb7f7b6b-32c8-489a-a020-1747c50fe84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:32.075693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 875/1778] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:32.076819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4787ace-cc32-40ae-9713-cd69e4adca7e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.086798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4787ace-cc32-40ae-9713-cd69e4adca7e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.091448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.092044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.092323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.092595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.093246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:32.093604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.095213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.107680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 904/1779] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:39:32.112178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f7b6b-32c8-489a-a020-1747c50fe84f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.122709 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f7b6b-32c8-489a-a020-1747c50fe84f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.128909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64d20b-4a6a-4a21-a267-9cac71e15426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.129531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64d20b-4a6a-4a21-a267-9cac71e15426 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.129811 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64d20b-4a6a-4a21-a267-9cac71e15426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.130134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64d20b-4a6a-4a21-a267-9cac71e15426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.133607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64d20b-4a6a-4a21-a267-9cac71e15426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.133607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64d20b-4a6a-4a21-a267-9cac71e15426 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.255702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 876/1780] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:32 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:32.259869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.269797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aff3da-354a-421b-b20a-eab08efd5fdb admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.328612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc662c7-c986-4f61-a44d-78f6afb763a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.329380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc662c7-c986-4f61-a44d-78f6afb763a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.329807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc662c7-c986-4f61-a44d-78f6afb763a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.330184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc662c7-c986-4f61-a44d-78f6afb763a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.330823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc662c7-c986-4f61-a44d-78f6afb763a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.332789 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc662c7-c986-4f61-a44d-78f6afb763a8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.421034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 905/1781] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:32 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:32.425246 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64d20b-4a6a-4a21-a267-9cac71e15426 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.435543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64d20b-4a6a-4a21-a267-9cac71e15426 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.440963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298e1d7-0f5d-4fda-ad56-db011658b358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.440963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e1d7-0f5d-4fda-ad56-db011658b358 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.441182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e1d7-0f5d-4fda-ad56-db011658b358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.441339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298e1d7-0f5d-4fda-ad56-db011658b358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.442005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298e1d7-0f5d-4fda-ad56-db011658b358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.443582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298e1d7-0f5d-4fda-ad56-db011658b358 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.449012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:32.455086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 877/1782] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:32.459461 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc662c7-c986-4f61-a44d-78f6afb763a8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.469493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc662c7-c986-4f61-a44d-78f6afb763a8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.474845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6ef1c-a3f1-44ff-8d9f-9ea1fa20290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.475435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ef1c-a3f1-44ff-8d9f-9ea1fa20290a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.475716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ef1c-a3f1-44ff-8d9f-9ea1fa20290a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.475990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6ef1c-a3f1-44ff-8d9f-9ea1fa20290a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.476657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6ef1c-a3f1-44ff-8d9f-9ea1fa20290a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.478282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6ef1c-a3f1-44ff-8d9f-9ea1fa20290a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.488719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 906/1783] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:32.492920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298e1d7-0f5d-4fda-ad56-db011658b358 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.502732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298e1d7-0f5d-4fda-ad56-db011658b358 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.507298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.507935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.508211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.508482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] PATH_INFO: `/v3/projects/a577af89db574335a91ac0ec31f29aaa/users/d84f616611084a019faa182c4257af2a/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.523418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a577af89db574335a91ac0ec31f29aaa, user_id=d84f616611084a019faa182c4257af2a, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.526261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.639125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 878/1784] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:32 2022] PUT /identity/v3/projects/a577af89db574335a91ac0ec31f29aaa/users/d84f616611084a019faa182c4257af2a/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:32.650140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb7875-a35b-4b0a-a607-b5961298b21b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.650635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb7875-a35b-4b0a-a607-b5961298b21b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.650759 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb7875-a35b-4b0a-a607-b5961298b21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.686496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b5cb7875-a35b-4b0a-a607-b5961298b21b None None] MFA Rules not processed for user `d84f616611084a019faa182c4257af2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:32.686496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b5cb7875-a35b-4b0a-a607-b5961298b21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:32.696676 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:32.761300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 907/1785] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:32.773344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.775163 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:32.850092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7af8e-5a99-4ff2-af4d-be89ec2564cc admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.854192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9b495-38ae-415c-8c70-658fa57eef5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.854810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b495-38ae-415c-8c70-658fa57eef5a service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.855093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b495-38ae-415c-8c70-658fa57eef5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.855510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b495-38ae-415c-8c70-658fa57eef5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.865291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9b495-38ae-415c-8c70-658fa57eef5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.867225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9b495-38ae-415c-8c70-658fa57eef5a service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.878100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 879/1786] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:32.901181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb7875-a35b-4b0a-a607-b5961298b21b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.972463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb7875-a35b-4b0a-a607-b5961298b21b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.977180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.977815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.978144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.978422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] PATH_INFO: `/v3/users/d84f616611084a019faa182c4257af2a` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.979611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84f616611084a019faa182c4257af2a)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.985331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:33.164833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e66b00-b40a-402d-b522-87e33f167224 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.164833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e66b00-b40a-402d-b522-87e33f167224 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.164833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e66b00-b40a-402d-b522-87e33f167224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.196027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e3e66b00-b40a-402d-b522-87e33f167224 None None] MFA Rules not processed for user `73f7c5ee59874429a4ecbee351702aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:33.197376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e3e66b00-b40a-402d-b522-87e33f167224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:33.232409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84f616611084a019faa182c4257af2a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:33.266506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84f616611084a019faa182c4257af2a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:33.267235 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84f616611084a019faa182c4257af2a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:33.273711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 880/1787] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:33 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:33.278350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e66b00-b40a-402d-b522-87e33f167224 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:33.280929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e3e66b00-b40a-402d-b522-87e33f167224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:33.296472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e66b00-b40a-402d-b522-87e33f167224 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:33.302157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:33.302894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.303278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.303683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.304423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:33.306624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:33.380896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84f616611084a019faa182c4257af2a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:33.388468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6e0b8857-0434-4940-9bf4-bca61c1c0bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84f616611084a019faa182c4257af2a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:33.394496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 908/1788] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:32 2022] DELETE /identity/v3/users/d84f616611084a019faa182c4257af2a => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:33.399029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae18e-4f7d-4774-9baa-652147393fea None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.399389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae18e-4f7d-4774-9baa-652147393fea None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.399733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ae18e-4f7d-4774-9baa-652147393fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.423600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 881/1789] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:33 2022] POST /identity/v3/projects => generated 405 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:33.423793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-691ae18e-4f7d-4774-9baa-652147393fea None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:33.424673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-691ae18e-4f7d-4774-9baa-652147393fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:33.424830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:33.426846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:33.484491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 909/1790] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:33.490626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2b15-ea8a-4481-a32f-cd7d31f184c6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:33.495294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ae18e-4f7d-4774-9baa-652147393fea None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:33.495649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f29f1f-51d6-4137-8417-5e31de15e0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:33.496257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29f1f-51d6-4137-8417-5e31de15e0bc admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.496533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29f1f-51d6-4137-8417-5e31de15e0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.496707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-691ae18e-4f7d-4774-9baa-652147393fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:33.496804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29f1f-51d6-4137-8417-5e31de15e0bc admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.497459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f29f1f-51d6-4137-8417-5e31de15e0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:33.499643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f29f1f-51d6-4137-8417-5e31de15e0bc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:33.563771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ae18e-4f7d-4774-9baa-652147393fea None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:33.567598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86988e7e-b583-4bb5-b338-2fb102b731bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:33.568211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86988e7e-b583-4bb5-b338-2fb102b731bc service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.568516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86988e7e-b583-4bb5-b338-2fb102b731bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.568846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86988e7e-b583-4bb5-b338-2fb102b731bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.578426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86988e7e-b583-4bb5-b338-2fb102b731bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:33.582323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86988e7e-b583-4bb5-b338-2fb102b731bc service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:33.589896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 910/1791] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:33.608161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 882/1792] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:33 2022] POST /identity/v3/users => generated 309 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:33.611881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86988e7e-b583-4bb5-b338-2fb102b731bc service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:33.624453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86988e7e-b583-4bb5-b338-2fb102b731bc service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:33.631427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e2fc4-0ad6-45a8-b2cb-ba81f8bbcd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:33.632553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e2fc4-0ad6-45a8-b2cb-ba81f8bbcd0c admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.635912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e2fc4-0ad6-45a8-b2cb-ba81f8bbcd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.635912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e2fc4-0ad6-45a8-b2cb-ba81f8bbcd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.635912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e2fc4-0ad6-45a8-b2cb-ba81f8bbcd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:33.636921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e2fc4-0ad6-45a8-b2cb-ba81f8bbcd0c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:33.654002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 911/1793] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:33.655961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f29f1f-51d6-4137-8417-5e31de15e0bc admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:33.666744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f29f1f-51d6-4137-8417-5e31de15e0bc admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:33.672010 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7432c-2cbe-427d-ab03-4d6c0ea8dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:33.672669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7432c-2cbe-427d-ab03-4d6c0ea8dd10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.672964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7432c-2cbe-427d-ab03-4d6c0ea8dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.673225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7432c-2cbe-427d-ab03-4d6c0ea8dd10 admin admin] PATH_INFO: `/v3/projects/402aa725c9e245dd999afc9092489093/users/0427998f278d4770a55875c4f0d80420/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.677031 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:33.689272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7432c-2cbe-427d-ab03-4d6c0ea8dd10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=402aa725c9e245dd999afc9092489093, user_id=0427998f278d4770a55875c4f0d80420, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:33.692002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7432c-2cbe-427d-ab03-4d6c0ea8dd10 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:33.771029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 883/1794] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:33 2022] PUT /identity/v3/projects/402aa725c9e245dd999afc9092489093/users/0427998f278d4770a55875c4f0d80420/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:33.777441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e2fc4-0ad6-45a8-b2cb-ba81f8bbcd0c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:33.804219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e2fc4-0ad6-45a8-b2cb-ba81f8bbcd0c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:33.878693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:33.879322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.879782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.880060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.880750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:33.882417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:33.896318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 912/1795] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:33.897845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7432c-2cbe-427d-ab03-4d6c0ea8dd10 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:33.910433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7432c-2cbe-427d-ab03-4d6c0ea8dd10 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:33.916276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ee8c9-31da-4598-8f31-26fda260e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:33.916961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee8c9-31da-4598-8f31-26fda260e80e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:33.917238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee8c9-31da-4598-8f31-26fda260e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:33.917512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ee8c9-31da-4598-8f31-26fda260e80e admin admin] PATH_INFO: `/v3/projects/402aa725c9e245dd999afc9092489093/users/0427998f278d4770a55875c4f0d80420/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:33.919577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ee8c9-31da-4598-8f31-26fda260e80e admin admin] RBAC: Authorizing `identity:create_grant(project_id=402aa725c9e245dd999afc9092489093, user_id=0427998f278d4770a55875c4f0d80420, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:33.922368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ee8c9-31da-4598-8f31-26fda260e80e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:33.934179 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 884/1796] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:33 2022] PUT /identity/v3/projects/402aa725c9e245dd999afc9092489093/users/0427998f278d4770a55875c4f0d80420/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:33.935800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:34.053438 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:34.055609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:34.078625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a6a7e-b653-431c-9dba-b1255089cc72 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:34.145571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:34.146276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.146558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.146828 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] PATH_INFO: `/v3/projects/a577af89db574335a91ac0ec31f29aaa` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.147958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a577af89db574335a91ac0ec31f29aaa)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:34.150043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:34.252054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a577af89db574335a91ac0ec31f29aaa', 'request_id': 'req-a77c0df9-01f7-4287-b141-0ef485232587'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:34.258247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a577af89db574335a91ac0ec31f29aaa', 'request_id': 'req-a77c0df9-01f7-4287-b141-0ef485232587'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:34.267473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a577af89db574335a91ac0ec31f29aaa', 'request_id': 'req-a77c0df9-01f7-4287-b141-0ef485232587'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:34.368554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] The token cache is being invalidate because project a577af89db574335a91ac0ec31f29aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:34.368922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:34.369623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a77c0df9-01f7-4287-b141-0ef485232587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:34.372608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 913/1797] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:34 2022] DELETE /identity/v3/projects/a577af89db574335a91ac0ec31f29aaa => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:34.376693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db57cc-bfbd-43fe-bd3e-8b8102436766 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.377075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db57cc-bfbd-43fe-bd3e-8b8102436766 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.377446 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db57cc-bfbd-43fe-bd3e-8b8102436766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.399106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d3db57cc-bfbd-43fe-bd3e-8b8102436766 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:34.400330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d3db57cc-bfbd-43fe-bd3e-8b8102436766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:34.461568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 885/1798] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:34.469229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c1924-e316-4e73-9331-0594b3cbe239 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.469516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c1924-e316-4e73-9331-0594b3cbe239 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.469795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c1924-e316-4e73-9331-0594b3cbe239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.489997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9d0c1924-e316-4e73-9331-0594b3cbe239 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:34.491201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9d0c1924-e316-4e73-9331-0594b3cbe239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:34.503114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 914/1799] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:34.504908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db57cc-bfbd-43fe-bd3e-8b8102436766 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:34.515012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db57cc-bfbd-43fe-bd3e-8b8102436766 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:34.520413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:34.520957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.521236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.521510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.522222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:34.522530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:34.524124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:34.536114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 886/1800] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:34.540305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c1924-e316-4e73-9331-0594b3cbe239 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:34.551058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c1924-e316-4e73-9331-0594b3cbe239 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:34.556645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee2bc6-cf0d-46e5-9b70-52fdf158dda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:34.557206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee2bc6-cf0d-46e5-9b70-52fdf158dda7 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.557683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee2bc6-cf0d-46e5-9b70-52fdf158dda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.558028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee2bc6-cf0d-46e5-9b70-52fdf158dda7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.567834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee2bc6-cf0d-46e5-9b70-52fdf158dda7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:34.571046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee2bc6-cf0d-46e5-9b70-52fdf158dda7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:34.673887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 915/1801] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:34 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:34.678401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:34.688444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ea4a-f008-4921-947c-11e219a5e659 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:34.747478 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1d7bb-04c7-4eca-9fe9-d6891047fa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:34.748161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1d7bb-04c7-4eca-9fe9-d6891047fa5b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.748579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1d7bb-04c7-4eca-9fe9-d6891047fa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.748955 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1d7bb-04c7-4eca-9fe9-d6891047fa5b admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.749660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1d7bb-04c7-4eca-9fe9-d6891047fa5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:34.751671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1d7bb-04c7-4eca-9fe9-d6891047fa5b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:34.831161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 887/1802] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:34 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:34.834513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee2bc6-cf0d-46e5-9b70-52fdf158dda7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:34.852681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee2bc6-cf0d-46e5-9b70-52fdf158dda7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:34.865179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41746236-dc04-4972-8b27-49ee171f314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:34.866517 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41746236-dc04-4972-8b27-49ee171f314c admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.867127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41746236-dc04-4972-8b27-49ee171f314c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.867410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41746236-dc04-4972-8b27-49ee171f314c admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.868190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41746236-dc04-4972-8b27-49ee171f314c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:34.872821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41746236-dc04-4972-8b27-49ee171f314c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:34.885462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 916/1803] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:34.908378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1d7bb-04c7-4eca-9fe9-d6891047fa5b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:34.918687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1d7bb-04c7-4eca-9fe9-d6891047fa5b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:34.947552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ff4bd-f65c-4562-ab25-4552c030926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:34.947552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff4bd-f65c-4562-ab25-4552c030926c admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.947552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff4bd-f65c-4562-ab25-4552c030926c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.947552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff4bd-f65c-4562-ab25-4552c030926c admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.948346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ff4bd-f65c-4562-ab25-4552c030926c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:34.948346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ff4bd-f65c-4562-ab25-4552c030926c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:34.948346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 888/1804] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:34.950747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41746236-dc04-4972-8b27-49ee171f314c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:34.961144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41746236-dc04-4972-8b27-49ee171f314c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:34.966423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac66eb-ae68-4852-b99b-2ced8b65d572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:34.967255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac66eb-ae68-4852-b99b-2ced8b65d572 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:34.967579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac66eb-ae68-4852-b99b-2ced8b65d572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:34.967853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac66eb-ae68-4852-b99b-2ced8b65d572 admin admin] PATH_INFO: `/v3/projects/5c324b4cd71a4a908615a43b451b54bd/users/be20c635816b4f82b7448ab07e05adf3/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:34.982869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac66eb-ae68-4852-b99b-2ced8b65d572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c324b4cd71a4a908615a43b451b54bd, user_id=be20c635816b4f82b7448ab07e05adf3, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:34.985514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac66eb-ae68-4852-b99b-2ced8b65d572 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:35.020812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:35.325467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 917/1805] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:34 2022] PUT /identity/v3/projects/5c324b4cd71a4a908615a43b451b54bd/users/be20c635816b4f82b7448ab07e05adf3/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:35.341559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ff69-9f6c-433a-a327-361ab10c9d1f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:35.342433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ff69-9f6c-433a-a327-361ab10c9d1f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:35.342433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ff69-9f6c-433a-a327-361ab10c9d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:35.352532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:35.380348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e1f2ff69-9f6c-433a-a327-361ab10c9d1f None None] MFA Rules not processed for user `be20c635816b4f82b7448ab07e05adf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:35.382173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e1f2ff69-9f6c-433a-a327-361ab10c9d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:35.487886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 889/1806] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:35.493101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac66eb-ae68-4852-b99b-2ced8b65d572 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:35.572427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac66eb-ae68-4852-b99b-2ced8b65d572 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:35.577513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86f391-fe8c-4d37-bf0b-df9bb018b302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:35.578102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86f391-fe8c-4d37-bf0b-df9bb018b302 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:35.578386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86f391-fe8c-4d37-bf0b-df9bb018b302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:35.578659 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86f391-fe8c-4d37-bf0b-df9bb018b302 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:35.579312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86f391-fe8c-4d37-bf0b-df9bb018b302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:35.581219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86f391-fe8c-4d37-bf0b-df9bb018b302 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:35.690954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c96b3-f25d-46a2-b266-12d66c6e92d2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:35.691792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c96b3-f25d-46a2-b266-12d66c6e92d2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:35.692066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c96b3-f25d-46a2-b266-12d66c6e92d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:35.705716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 918/1807] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:35 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:35.710603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86f391-fe8c-4d37-bf0b-df9bb018b302 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:35.724056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:35.737170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-296c96b3-f25d-46a2-b266-12d66c6e92d2 None None] MFA Rules not processed for user `0427998f278d4770a55875c4f0d80420`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:35.739235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-296c96b3-f25d-46a2-b266-12d66c6e92d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:35.815643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86f391-fe8c-4d37-bf0b-df9bb018b302 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:35.821759 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019922d-3cdb-4db7-83bf-1df66aa2da52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:35.822679 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019922d-3cdb-4db7-83bf-1df66aa2da52 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:35.823184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019922d-3cdb-4db7-83bf-1df66aa2da52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:35.823617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019922d-3cdb-4db7-83bf-1df66aa2da52 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:35.824547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019922d-3cdb-4db7-83bf-1df66aa2da52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:35.827335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019922d-3cdb-4db7-83bf-1df66aa2da52 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:35.866453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 890/1808] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:35 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:35.931283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 919/1809] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:35 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:35.933195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c96b3-f25d-46a2-b266-12d66c6e92d2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:35.960473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c96b3-f25d-46a2-b266-12d66c6e92d2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:35.971269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1eb10-8f8d-4a2d-b033-0cdabd1a0e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:35.971981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1eb10-8f8d-4a2d-b033-0cdabd1a0e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:35.972464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1eb10-8f8d-4a2d-b033-0cdabd1a0e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:35.972915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1eb10-8f8d-4a2d-b033-0cdabd1a0e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:35.973671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1eb10-8f8d-4a2d-b033-0cdabd1a0e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:35.981714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1eb10-8f8d-4a2d-b033-0cdabd1a0e44 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:36.000482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 891/1810] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:36.005217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019922d-3cdb-4db7-83bf-1df66aa2da52 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:36.015827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019922d-3cdb-4db7-83bf-1df66aa2da52 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:36.022185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebe817-be9f-480e-b954-4fda353e3353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:36.022883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebe817-be9f-480e-b954-4fda353e3353 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.023237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebe817-be9f-480e-b954-4fda353e3353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.023584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebe817-be9f-480e-b954-4fda353e3353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.024359 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebe817-be9f-480e-b954-4fda353e3353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:36.026225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebe817-be9f-480e-b954-4fda353e3353 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:36.036618 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 920/1811] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:36.040780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1eb10-8f8d-4a2d-b033-0cdabd1a0e44 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:36.051637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1eb10-8f8d-4a2d-b033-0cdabd1a0e44 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:36.056431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c2100-441b-4ebd-895b-0bb089f8614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:36.057090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2100-441b-4ebd-895b-0bb089f8614b admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.057390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2100-441b-4ebd-895b-0bb089f8614b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.057648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2100-441b-4ebd-895b-0bb089f8614b admin admin] PATH_INFO: `/v3/projects/1993472c5aea4e2f8edb21b4f1a54f1f/users/bbf63eb44e2a4e22bbd087850cd4cb56/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.074706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c2100-441b-4ebd-895b-0bb089f8614b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1993472c5aea4e2f8edb21b4f1a54f1f, user_id=bbf63eb44e2a4e22bbd087850cd4cb56, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:36.077625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c2100-441b-4ebd-895b-0bb089f8614b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:36.167685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 892/1812] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:36 2022] PUT /identity/v3/projects/1993472c5aea4e2f8edb21b4f1a54f1f/users/bbf63eb44e2a4e22bbd087850cd4cb56/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:36.175131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf271c55-6f68-426f-99f2-ba96962588a0 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.175933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf271c55-6f68-426f-99f2-ba96962588a0 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.176221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf271c55-6f68-426f-99f2-ba96962588a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.207243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-cf271c55-6f68-426f-99f2-ba96962588a0 None None] MFA Rules not processed for user `bbf63eb44e2a4e22bbd087850cd4cb56`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:36.208469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-cf271c55-6f68-426f-99f2-ba96962588a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:36.278313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 921/1813] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:36.283017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c2100-441b-4ebd-895b-0bb089f8614b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:36.341480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:36.378061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c2100-441b-4ebd-895b-0bb089f8614b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:36.382845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90961d-d8ab-4136-9637-ec45be571995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:36.383486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90961d-d8ab-4136-9637-ec45be571995 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.383780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90961d-d8ab-4136-9637-ec45be571995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.384028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90961d-d8ab-4136-9637-ec45be571995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.384690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90961d-d8ab-4136-9637-ec45be571995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:36.387306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90961d-d8ab-4136-9637-ec45be571995 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:36.515653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 893/1814] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:36 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:36.519937 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf271c55-6f68-426f-99f2-ba96962588a0 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:36.531670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf271c55-6f68-426f-99f2-ba96962588a0 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:36.607680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17faf2a3-5668-4d65-bc32-1fbb870d5101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:36.608491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faf2a3-5668-4d65-bc32-1fbb870d5101 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.608836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faf2a3-5668-4d65-bc32-1fbb870d5101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.609213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faf2a3-5668-4d65-bc32-1fbb870d5101 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.610152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17faf2a3-5668-4d65-bc32-1fbb870d5101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:36.612166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17faf2a3-5668-4d65-bc32-1fbb870d5101 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:36.685839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b969b6-a490-4857-b6b7-ad6bd8b77a09 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.686195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b969b6-a490-4857-b6b7-ad6bd8b77a09 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.686618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b969b6-a490-4857-b6b7-ad6bd8b77a09 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.688345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 894/1815] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:39:36.706819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 922/1816] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:36 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:36.713607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b969b6-a490-4857-b6b7-ad6bd8b77a09 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:36.724663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd92a5-9631-45cd-aebe-7bc0bcb36201 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.724936 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd92a5-9631-45cd-aebe-7bc0bcb36201 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.725207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd92a5-9631-45cd-aebe-7bc0bcb36201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.729124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b969b6-a490-4857-b6b7-ad6bd8b77a09 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:36.735546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:36.737014 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.737512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.737859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.747983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:36.747983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:36.760670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 895/1817] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:36.761861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:36.763612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:36.765282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a9bd92a5-9631-45cd-aebe-7bc0bcb36201 None None] MFA Rules not processed for user `f35e7add10924dc49eb9d4a983c89eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:36.766829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a9bd92a5-9631-45cd-aebe-7bc0bcb36201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:36.782716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9acf0-e00c-4607-a52c-17119c4c9da0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:36.789413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:36.790147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.790671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.790987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] PATH_INFO: `/v3/users/79d9d641e9bf4570afe85aeb9662f110` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.792432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=79d9d641e9bf4570afe85aeb9662f110)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:36.794808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:36.844231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 923/1818] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:39:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:36.845330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd92a5-9631-45cd-aebe-7bc0bcb36201 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:36.857656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd92a5-9631-45cd-aebe-7bc0bcb36201 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:36.862262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:36.863593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:36.863872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:36.864141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] PATH_INFO: `/v3/projects/a6410a34e84a475c9583f2f399baf2c2/users/b37864f7329f47729a2b3fb5823c38d0/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:36.879833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6410a34e84a475c9583f2f399baf2c2, user_id=b37864f7329f47729a2b3fb5823c38d0, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:36.883437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:36.928238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:36.992967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 924/1819] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:36 2022] PUT /identity/v3/projects/a6410a34e84a475c9583f2f399baf2c2/users/b37864f7329f47729a2b3fb5823c38d0/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:36.993839 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:36.995586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:37.032047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79d9d641e9bf4570afe85aeb9662f110'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.046295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79d9d641e9bf4570afe85aeb9662f110'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.073580 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649c8db-97ed-45ed-855d-bea99cfa7684 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:37.078935 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a570b0-1404-4232-b6a7-2e1ff2413405 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:37.080636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a570b0-1404-4232-b6a7-2e1ff2413405 service placement] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:37.080636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a570b0-1404-4232-b6a7-2e1ff2413405 service placement] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:37.080636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a570b0-1404-4232-b6a7-2e1ff2413405 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:37.089467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a570b0-1404-4232-b6a7-2e1ff2413405 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:37.091852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a570b0-1404-4232-b6a7-2e1ff2413405 service placement] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:37.102560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 925/1820] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:37.103376 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a570b0-1404-4232-b6a7-2e1ff2413405 service placement] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:37.122701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79d9d641e9bf4570afe85aeb9662f110'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.123350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79d9d641e9bf4570afe85aeb9662f110'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.132052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79d9d641e9bf4570afe85aeb9662f110'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.141904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 896/1821] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:36 2022] DELETE /identity/v3/users/79d9d641e9bf4570afe85aeb9662f110 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:37.174050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a570b0-1404-4232-b6a7-2e1ff2413405 service placement] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:37.237204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398200df-4bdd-4868-80cd-165ac277d8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:37.237972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398200df-4bdd-4868-80cd-165ac277d8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:37.238324 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398200df-4bdd-4868-80cd-165ac277d8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:37.238668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398200df-4bdd-4868-80cd-165ac277d8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:37.239436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398200df-4bdd-4868-80cd-165ac277d8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:37.240980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398200df-4bdd-4868-80cd-165ac277d8a4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:37.252331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 926/1822] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:37.256530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:37.258226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:37.272595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdb2e0-6cb2-420a-8e98-662b3d401caf admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:37.280032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:37.286870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:37.286870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:37.286870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] PATH_INFO: `/v3/projects/a6410a34e84a475c9583f2f399baf2c2/users/b37864f7329f47729a2b3fb5823c38d0/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:37.286870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6410a34e84a475c9583f2f399baf2c2, user_id=b37864f7329f47729a2b3fb5823c38d0, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:37.287734 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:37.304766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 897/1823] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:37 2022] PUT /identity/v3/projects/a6410a34e84a475c9583f2f399baf2c2/users/b37864f7329f47729a2b3fb5823c38d0/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:37.310885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ca1d4-534a-4ab4-a7bc-7bf4be5f8e2b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:37.311561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ca1d4-534a-4ab4-a7bc-7bf4be5f8e2b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:37.311987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ca1d4-534a-4ab4-a7bc-7bf4be5f8e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:37.344637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-6b9ca1d4-534a-4ab4-a7bc-7bf4be5f8e2b None None] MFA Rules not processed for user `b37864f7329f47729a2b3fb5823c38d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:37.346098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6b9ca1d4-534a-4ab4-a7bc-7bf4be5f8e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:37.425710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 927/1824] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:37.436095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:37.438537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:37.446993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ca1d4-534a-4ab4-a7bc-7bf4be5f8e2b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:37.448691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6b9ca1d4-534a-4ab4-a7bc-7bf4be5f8e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:37.528076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfdf9d-3a20-4304-9588-21bb87234d35 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:37.533211 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:37.533747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ca1d4-534a-4ab4-a7bc-7bf4be5f8e2b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:37.533893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:37.534143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:37.534422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] PATH_INFO: `/v3/projects/56014854c0694a1498a455f64bb4a194` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:37.535662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=56014854c0694a1498a455f64bb4a194)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:37.537627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:37.538066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:37.538738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:37.539230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:37.539541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:37.541750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:37.560419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:37.610523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:37.611984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:37.622590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 928/1825] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:37.626889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:37.629054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:37.654483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d97ae3-b2ac-43f3-970d-5cf128e59afd service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:37.655154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56014854c0694a1498a455f64bb4a194', 'request_id': 'req-2a6e7237-13a2-404b-a854-27a9e4db7a1e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.661359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56014854c0694a1498a455f64bb4a194', 'request_id': 'req-2a6e7237-13a2-404b-a854-27a9e4db7a1e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.669421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56014854c0694a1498a455f64bb4a194', 'request_id': 'req-2a6e7237-13a2-404b-a854-27a9e4db7a1e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.713812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:37.714624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:37.714938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:37.715214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] PATH_INFO: `/v3/users/73f7c5ee59874429a4ecbee351702aa0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:37.716470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f7c5ee59874429a4ecbee351702aa0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:37.718517 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:37.746262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] The token cache is being invalidate because project 56014854c0694a1498a455f64bb4a194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:37.746807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.754140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.754140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 898/1826] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:37 2022] DELETE /identity/v3/projects/56014854c0694a1498a455f64bb4a194 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:37.754140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:37.755145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:37.767670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e7237-13a2-404b-a854-27a9e4db7a1e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:37.772404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:37.773019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:37.773300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:37.773568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] PATH_INFO: `/v3/users/b9a0ec148c2043c9ad8b88964948e693` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:37.774724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a0ec148c2043c9ad8b88964948e693)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:37.777005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:37.916970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f7c5ee59874429a4ecbee351702aa0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.931472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f7c5ee59874429a4ecbee351702aa0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.932232 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f7c5ee59874429a4ecbee351702aa0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.936203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a0ec148c2043c9ad8b88964948e693'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.962893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a0ec148c2043c9ad8b88964948e693'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.964018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:37.986453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f7c5ee59874429a4ecbee351702aa0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:37.995259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f7c5ee59874429a4ecbee351702aa0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.001271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 929/1827] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:37 2022] DELETE /identity/v3/users/73f7c5ee59874429a4ecbee351702aa0 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:38.071771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a0ec148c2043c9ad8b88964948e693'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.072558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a0ec148c2043c9ad8b88964948e693'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.080668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a0ec148c2043c9ad8b88964948e693'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.087396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 899/1828] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:37 2022] DELETE /identity/v3/users/b9a0ec148c2043c9ad8b88964948e693 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:38.272302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:38.273732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:38.352551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e773cd-69d3-4eef-a14e-cdc9537343b1 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:38.360706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:38.362601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:38.433302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:38.434493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:38.434809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:38.435219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] PATH_INFO: `/v3/projects/c87dc51bd48d4f72a26ae2fd676ff73f` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:38.437576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c87dc51bd48d4f72a26ae2fd676ff73f)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:38.439637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:38.466931 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba20e4-df97-43f1-9907-eee08b7a7e34 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:38.472095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:38.473821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:38.474125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:38.474396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] PATH_INFO: `/v3/projects/9d6cfd5de8814a75bebb1d73ab9c6a60` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:38.475533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6cfd5de8814a75bebb1d73ab9c6a60)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:38.477480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:38.569147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6cfd5de8814a75bebb1d73ab9c6a60', 'request_id': 'req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.570248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c87dc51bd48d4f72a26ae2fd676ff73f', 'request_id': 'req-46f4b9cb-20a5-414d-b224-200b637c6c3d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.575758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c87dc51bd48d4f72a26ae2fd676ff73f', 'request_id': 'req-46f4b9cb-20a5-414d-b224-200b637c6c3d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.576833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6cfd5de8814a75bebb1d73ab9c6a60', 'request_id': 'req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.582771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c87dc51bd48d4f72a26ae2fd676ff73f', 'request_id': 'req-46f4b9cb-20a5-414d-b224-200b637c6c3d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.584467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6cfd5de8814a75bebb1d73ab9c6a60', 'request_id': 'req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.652795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] The token cache is being invalidate because project c87dc51bd48d4f72a26ae2fd676ff73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:38.653088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.653714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.655367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 930/1829] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:38 2022] DELETE /identity/v3/projects/c87dc51bd48d4f72a26ae2fd676ff73f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:38.659278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:38.661212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:38.723388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4b9cb-20a5-414d-b224-200b637c6c3d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:38.728078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:38.728701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:38.728979 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:38.729258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] PATH_INFO: `/v3/users/0427998f278d4770a55875c4f0d80420` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:38.730434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0427998f278d4770a55875c4f0d80420)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:38.732346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:38.743017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] The token cache is being invalidate because project 9d6cfd5de8814a75bebb1d73ab9c6a60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:38.743017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.743017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-637d8fbd-5e18-4c8c-8a12-a559df03ee0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.743017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 900/1830] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:38 2022] DELETE /identity/v3/projects/9d6cfd5de8814a75bebb1d73ab9c6a60 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:38.743017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81878f-8cf8-4f98-8dd4-a8e9ec254104 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:38.743554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81878f-8cf8-4f98-8dd4-a8e9ec254104 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:38.743554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81878f-8cf8-4f98-8dd4-a8e9ec254104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:38.761974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-eb81878f-8cf8-4f98-8dd4-a8e9ec254104 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:38.763214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-eb81878f-8cf8-4f98-8dd4-a8e9ec254104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:38.772168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 901/1831] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:38.781689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20316e32-0c83-4ec6-823f-36fca939f83f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:38.782003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20316e32-0c83-4ec6-823f-36fca939f83f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:38.782283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20316e32-0c83-4ec6-823f-36fca939f83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:38.801548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-20316e32-0c83-4ec6-823f-36fca939f83f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:38.802778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-20316e32-0c83-4ec6-823f-36fca939f83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:38.810923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 902/1832] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:38.815037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20316e32-0c83-4ec6-823f-36fca939f83f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:38.825509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20316e32-0c83-4ec6-823f-36fca939f83f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:38.830578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:38.831177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:38.831455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:38.832300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:38.832978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:38.833340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:38.834973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:38.838921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:38.847474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 903/1833] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:39:38.851507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:38.861050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06365e0-3133-42d5-9f2d-3265a7a8fe1b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:38.865742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:38.866376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:38.866651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:38.867202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:38.867825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:38.869669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:38.886692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0427998f278d4770a55875c4f0d80420'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.899947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0427998f278d4770a55875c4f0d80420'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.900668 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0427998f278d4770a55875c4f0d80420'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.935752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 904/1834] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:38 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:38.939865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:38.941922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:38.950419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0427998f278d4770a55875c4f0d80420'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.953005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:38.958392 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0427998f278d4770a55875c4f0d80420'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:38.964337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 931/1835] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:38 2022] DELETE /identity/v3/users/0427998f278d4770a55875c4f0d80420 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:39.009713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3753c-faf7-4218-a903-d93e93d82794 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.084020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc265ce4-0509-4558-9afa-ce86ef92a6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.084640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc265ce4-0509-4558-9afa-ce86ef92a6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.084918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc265ce4-0509-4558-9afa-ce86ef92a6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.085194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc265ce4-0509-4558-9afa-ce86ef92a6c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.085829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc265ce4-0509-4558-9afa-ce86ef92a6c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.087984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc265ce4-0509-4558-9afa-ce86ef92a6c7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.088901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:39.209972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 905/1836] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:38 2022] POST /identity/v3/users => generated 289 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:39.212939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.225269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fefe29-0188-4afe-b866-d3735b84f4fc admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.230906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b52bc0-1e45-4ba5-865c-7345357135ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.231572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b52bc0-1e45-4ba5-865c-7345357135ea admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.231863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b52bc0-1e45-4ba5-865c-7345357135ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.232142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b52bc0-1e45-4ba5-865c-7345357135ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.232819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b52bc0-1e45-4ba5-865c-7345357135ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.234557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b52bc0-1e45-4ba5-865c-7345357135ea admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.245731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 932/1837] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:39.249641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc265ce4-0509-4558-9afa-ce86ef92a6c7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.260695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc265ce4-0509-4558-9afa-ce86ef92a6c7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.265350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.280243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.280243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.280243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.280243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.280243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.280243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 906/1838] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:39.283216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b52bc0-1e45-4ba5-865c-7345357135ea admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.293301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b52bc0-1e45-4ba5-865c-7345357135ea admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.298115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aaad57-cf7d-4611-bb3b-e5397b33ecb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.298772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaad57-cf7d-4611-bb3b-e5397b33ecb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.299079 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaad57-cf7d-4611-bb3b-e5397b33ecb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.299307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aaad57-cf7d-4611-bb3b-e5397b33ecb5 admin admin] PATH_INFO: `/v3/projects/9270d8acba3e49028f0e3da555172c8f/users/fa54a46f4fe14ca39c20e10b258cf8f7/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.306882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.309389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:39.315554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aaad57-cf7d-4611-bb3b-e5397b33ecb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9270d8acba3e49028f0e3da555172c8f, user_id=fa54a46f4fe14ca39c20e10b258cf8f7, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.320056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aaad57-cf7d-4611-bb3b-e5397b33ecb5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.325274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf43c5-b47d-4600-8288-d770cf6b5a63 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.330115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.330743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.331020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.331888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] PATH_INFO: `/v3/projects/402aa725c9e245dd999afc9092489093` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.333464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=402aa725c9e245dd999afc9092489093)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.335358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.484045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 933/1839] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:39 2022] PUT /identity/v3/projects/9270d8acba3e49028f0e3da555172c8f/users/fa54a46f4fe14ca39c20e10b258cf8f7/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:39.495910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5573190-ec97-4ce4-b858-c85e564c4810 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.496196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5573190-ec97-4ce4-b858-c85e564c4810 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.496472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5573190-ec97-4ce4-b858-c85e564c4810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.522995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '402aa725c9e245dd999afc9092489093', 'request_id': 'req-75712552-f0aa-4b4c-831d-dba62be94386'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.524930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c5573190-ec97-4ce4-b858-c85e564c4810 None None] MFA Rules not processed for user `fa54a46f4fe14ca39c20e10b258cf8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:39.526342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c5573190-ec97-4ce4-b858-c85e564c4810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:39.528719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '402aa725c9e245dd999afc9092489093', 'request_id': 'req-75712552-f0aa-4b4c-831d-dba62be94386'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.536089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '402aa725c9e245dd999afc9092489093', 'request_id': 'req-75712552-f0aa-4b4c-831d-dba62be94386'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.589265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 934/1840] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:39.593684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5573190-ec97-4ce4-b858-c85e564c4810 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.655645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5573190-ec97-4ce4-b858-c85e564c4810 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.660888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] The token cache is being invalidate because project 402aa725c9e245dd999afc9092489093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:39.661152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.661318 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.661741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.661860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-75712552-f0aa-4b4c-831d-dba62be94386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.662043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.662312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.662936 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.663637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 907/1841] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:39 2022] DELETE /identity/v3/projects/402aa725c9e245dd999afc9092489093 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:39.664827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.667848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:39.669524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ef65f-013f-4b57-981a-1aaba5a12a4a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.669823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ef65f-013f-4b57-981a-1aaba5a12a4a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.670717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ef65f-013f-4b57-981a-1aaba5a12a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.698138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-160ef65f-013f-4b57-981a-1aaba5a12a4a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:39.699347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-160ef65f-013f-4b57-981a-1aaba5a12a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:39.707264 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 908/1842] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:39.717489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b908de-42fb-4b90-ac78-0810e90d6121 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.717771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b908de-42fb-4b90-ac78-0810e90d6121 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.718088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b908de-42fb-4b90-ac78-0810e90d6121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.728178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:39.737868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f3b908de-42fb-4b90-ac78-0810e90d6121 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:39.739784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f3b908de-42fb-4b90-ac78-0810e90d6121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:39.748249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 909/1843] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:39.752225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b908de-42fb-4b90-ac78-0810e90d6121 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.761303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b908de-42fb-4b90-ac78-0810e90d6121 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.765340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 935/1844] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:39.766149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.766743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.767026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.767302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.767968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:39.768245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.769312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.769848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.770903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:39.786076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 910/1845] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:39:39.787074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.796726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c1809-203c-4d0e-83a0-4f91fa4a79ce admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.835538 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197054b-a5fb-4609-9827-b752e0f1dfef admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.840299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f01d2-1042-4aef-98d2-1df80959d208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.840898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f01d2-1042-4aef-98d2-1df80959d208 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.841176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f01d2-1042-4aef-98d2-1df80959d208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.841444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f01d2-1042-4aef-98d2-1df80959d208 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.842151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f01d2-1042-4aef-98d2-1df80959d208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.844117 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f01d2-1042-4aef-98d2-1df80959d208 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.856022 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edfd26-9d05-4d76-b03d-5c44032fa05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.856727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edfd26-9d05-4d76-b03d-5c44032fa05c admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.857182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edfd26-9d05-4d76-b03d-5c44032fa05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.860086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edfd26-9d05-4d76-b03d-5c44032fa05c admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.860086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8edfd26-9d05-4d76-b03d-5c44032fa05c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.860276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8edfd26-9d05-4d76-b03d-5c44032fa05c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.935241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 936/1846] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:39.938933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f01d2-1042-4aef-98d2-1df80959d208 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.948550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f01d2-1042-4aef-98d2-1df80959d208 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.949967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 911/1847] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/projects => generated 393 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:39.954292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edfd26-9d05-4d76-b03d-5c44032fa05c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.986238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:40.005956 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f155a5-88b7-4898-8055-f439763dfdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.006571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f155a5-88b7-4898-8055-f439763dfdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.006862 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f155a5-88b7-4898-8055-f439763dfdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.007128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f155a5-88b7-4898-8055-f439763dfdb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.007895 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f155a5-88b7-4898-8055-f439763dfdb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.009736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f155a5-88b7-4898-8055-f439763dfdb3 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.012364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:40.016482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edfd26-9d05-4d76-b03d-5c44032fa05c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.021650 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e559c-8e4a-4c6a-9d45-5889fec88093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.022277 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e559c-8e4a-4c6a-9d45-5889fec88093 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.022375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 937/1848] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:40.022562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e559c-8e4a-4c6a-9d45-5889fec88093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.022826 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e559c-8e4a-4c6a-9d45-5889fec88093 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.023481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e559c-8e4a-4c6a-9d45-5889fec88093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.025463 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e559c-8e4a-4c6a-9d45-5889fec88093 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.026316 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f155a5-88b7-4898-8055-f439763dfdb3 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.036716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f155a5-88b7-4898-8055-f439763dfdb3 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.041270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621f50b-2640-4d62-a74b-6f660e1c8646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.041848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621f50b-2640-4d62-a74b-6f660e1c8646 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.042154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621f50b-2640-4d62-a74b-6f660e1c8646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.042422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621f50b-2640-4d62-a74b-6f660e1c8646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.043046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621f50b-2640-4d62-a74b-6f660e1c8646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.044601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621f50b-2640-4d62-a74b-6f660e1c8646 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.055152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 938/1849] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:40.059105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621f50b-2640-4d62-a74b-6f660e1c8646 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.068549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621f50b-2640-4d62-a74b-6f660e1c8646 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.073173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276c65e-1004-469b-b1d0-29cf91739147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.076042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c65e-1004-469b-b1d0-29cf91739147 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.076042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c65e-1004-469b-b1d0-29cf91739147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.076042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276c65e-1004-469b-b1d0-29cf91739147 admin admin] PATH_INFO: `/v3/projects/9ee0e936b25f4c93afcd34e34a75c206/users/8a4480b703944a349a1ad100b3acc84c/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.089664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276c65e-1004-469b-b1d0-29cf91739147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee0e936b25f4c93afcd34e34a75c206, user_id=8a4480b703944a349a1ad100b3acc84c, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.092335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276c65e-1004-469b-b1d0-29cf91739147 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.129389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 912/1850] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:40.133638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e559c-8e4a-4c6a-9d45-5889fec88093 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.142870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e559c-8e4a-4c6a-9d45-5889fec88093 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.148309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f58c5-f52e-4b59-8efc-25cc52029c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.148991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f58c5-f52e-4b59-8efc-25cc52029c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.149276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f58c5-f52e-4b59-8efc-25cc52029c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.149595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f58c5-f52e-4b59-8efc-25cc52029c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.150289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f58c5-f52e-4b59-8efc-25cc52029c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.151969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f58c5-f52e-4b59-8efc-25cc52029c34 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.173642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 913/1851] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:40.173642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f58c5-f52e-4b59-8efc-25cc52029c34 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.179702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 939/1852] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:40 2022] PUT /identity/v3/projects/9ee0e936b25f4c93afcd34e34a75c206/users/8a4480b703944a349a1ad100b3acc84c/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:40.184159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f58c5-f52e-4b59-8efc-25cc52029c34 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.188403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c7d17-3371-496c-8d74-9b99cbf90ecd None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.188682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c7d17-3371-496c-8d74-9b99cbf90ecd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.188950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c7d17-3371-496c-8d74-9b99cbf90ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.217802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-718c7d17-3371-496c-8d74-9b99cbf90ecd None None] MFA Rules not processed for user `8a4480b703944a349a1ad100b3acc84c`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:40.219070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-718c7d17-3371-496c-8d74-9b99cbf90ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:40.246145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a929e7-fa8f-4cac-957a-4421d6172f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.246802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a929e7-fa8f-4cac-957a-4421d6172f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.247088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a929e7-fa8f-4cac-957a-4421d6172f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.247382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a929e7-fa8f-4cac-957a-4421d6172f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.248237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a929e7-fa8f-4cac-957a-4421d6172f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.250363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a929e7-fa8f-4cac-957a-4421d6172f6e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.266108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 914/1853] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:40.266778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a929e7-fa8f-4cac-957a-4421d6172f6e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.278057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a929e7-fa8f-4cac-957a-4421d6172f6e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.280253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:40.283343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a81c92-260e-445b-8b36-2369dd5915c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.288462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a81c92-260e-445b-8b36-2369dd5915c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.288462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a81c92-260e-445b-8b36-2369dd5915c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.288462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a81c92-260e-445b-8b36-2369dd5915c5 admin admin] PATH_INFO: `/v3/projects/000745d932fa489284f90333ec0fbbdb/users/87829e595f504195bd95fedb5648e31a/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.296868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 940/1854] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:40.300772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a81c92-260e-445b-8b36-2369dd5915c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=000745d932fa489284f90333ec0fbbdb, user_id=87829e595f504195bd95fedb5648e31a, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.301537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c7d17-3371-496c-8d74-9b99cbf90ecd None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.303513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a81c92-260e-445b-8b36-2369dd5915c5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.312727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c7d17-3371-496c-8d74-9b99cbf90ecd None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.318135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0303a05-6b3c-4a75-a5ff-6fd5f05e10e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.318827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0303a05-6b3c-4a75-a5ff-6fd5f05e10e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.319210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0303a05-6b3c-4a75-a5ff-6fd5f05e10e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.319558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0303a05-6b3c-4a75-a5ff-6fd5f05e10e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.320189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0303a05-6b3c-4a75-a5ff-6fd5f05e10e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.322124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0303a05-6b3c-4a75-a5ff-6fd5f05e10e8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.370016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 915/1855] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:40 2022] PUT /identity/v3/projects/000745d932fa489284f90333ec0fbbdb/users/87829e595f504195bd95fedb5648e31a/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:40.380644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f6d9-090e-4dd0-bd2c-caf7243b31e6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.380951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f6d9-090e-4dd0-bd2c-caf7243b31e6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.381198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f6d9-090e-4dd0-bd2c-caf7243b31e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.410439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f1f9f6d9-090e-4dd0-bd2c-caf7243b31e6 None None] MFA Rules not processed for user `87829e595f504195bd95fedb5648e31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:40.411653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f1f9f6d9-090e-4dd0-bd2c-caf7243b31e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:40.430374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 941/1856] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:40 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:40.435483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0303a05-6b3c-4a75-a5ff-6fd5f05e10e8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.479990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 916/1857] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:40.484366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f6d9-090e-4dd0-bd2c-caf7243b31e6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.499340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:40.545658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0303a05-6b3c-4a75-a5ff-6fd5f05e10e8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.551193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a6e66-5eba-42bc-bccf-8ac4e252b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.551940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6e66-5eba-42bc-bccf-8ac4e252b4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.552354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6e66-5eba-42bc-bccf-8ac4e252b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.552751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6e66-5eba-42bc-bccf-8ac4e252b4f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.553746 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a6e66-5eba-42bc-bccf-8ac4e252b4f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.555545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a6e66-5eba-42bc-bccf-8ac4e252b4f2 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.557420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f6d9-090e-4dd0-bd2c-caf7243b31e6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.562868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78521e3a-6c8c-4a87-a177-ddd7bcc7d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.563474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78521e3a-6c8c-4a87-a177-ddd7bcc7d675 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.563752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78521e3a-6c8c-4a87-a177-ddd7bcc7d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.564024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78521e3a-6c8c-4a87-a177-ddd7bcc7d675 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.564657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78521e3a-6c8c-4a87-a177-ddd7bcc7d675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.567142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78521e3a-6c8c-4a87-a177-ddd7bcc7d675 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.676410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 917/1858] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:40 2022] POST /identity/v3/projects => generated 393 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:40.683845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78521e3a-6c8c-4a87-a177-ddd7bcc7d675 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.684299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 942/1859] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:40 2022] POST /identity/v3/users => generated 295 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:40.687918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a6e66-5eba-42bc-bccf-8ac4e252b4f2 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.716156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a6e66-5eba-42bc-bccf-8ac4e252b4f2 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.760777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78521e3a-6c8c-4a87-a177-ddd7bcc7d675 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.765671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e136453-8e6a-4e5a-be59-167f4c0b1ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.766432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e136453-8e6a-4e5a-be59-167f4c0b1ecc admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.766984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e136453-8e6a-4e5a-be59-167f4c0b1ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.766984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e136453-8e6a-4e5a-be59-167f4c0b1ecc admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.767604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e136453-8e6a-4e5a-be59-167f4c0b1ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.769545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e136453-8e6a-4e5a-be59-167f4c0b1ecc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.786547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.787152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.787429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.789114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.789114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.789982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.799767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 943/1860] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:40.801083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.802652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:40.844021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 918/1861] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:40 2022] POST /identity/v3/users => generated 303 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:40.844827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e136453-8e6a-4e5a-be59-167f4c0b1ecc admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.856075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e136453-8e6a-4e5a-be59-167f4c0b1ecc admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.857072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:40.860901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50ac4d-b24f-46a6-80f6-c80092c6bd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.861074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513d20b-d9da-4ea3-8abb-a39081c68d59 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.861551 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ac4d-b24f-46a6-80f6-c80092c6bd52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.861829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ac4d-b24f-46a6-80f6-c80092c6bd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.862138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ac4d-b24f-46a6-80f6-c80092c6bd52 admin admin] PATH_INFO: `/v3/projects/07c4b472d8a84d33ae248c2dc1c4c010/users/4dd00a0a2d2f4a018782da0ca475e59b/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.866535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.867245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.867527 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.867801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.869597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:40.879925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50ac4d-b24f-46a6-80f6-c80092c6bd52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c4b472d8a84d33ae248c2dc1c4c010, user_id=4dd00a0a2d2f4a018782da0ca475e59b, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.882663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50ac4d-b24f-46a6-80f6-c80092c6bd52 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.929444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.930920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.939911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 944/1862] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:40.942636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.950964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40a212-6e40-4f98-a362-e24aa8ef06e6 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.955913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e7280-d1eb-4daa-af4e-b380720c0f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.956531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e7280-d1eb-4daa-af4e-b380720c0f73 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.956824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e7280-d1eb-4daa-af4e-b380720c0f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.957098 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e7280-d1eb-4daa-af4e-b380720c0f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.957723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e7280-d1eb-4daa-af4e-b380720c0f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.959315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e7280-d1eb-4daa-af4e-b380720c0f73 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.969609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 945/1863] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:40.969777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 919/1864] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:40 2022] PUT /identity/v3/projects/07c4b472d8a84d33ae248c2dc1c4c010/users/4dd00a0a2d2f4a018782da0ca475e59b/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:40.973782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e7280-d1eb-4daa-af4e-b380720c0f73 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.974040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50ac4d-b24f-46a6-80f6-c80092c6bd52 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.985109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e7280-d1eb-4daa-af4e-b380720c0f73 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:41.048256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50ac4d-b24f-46a6-80f6-c80092c6bd52 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:41.051422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546fedc-2dfb-4c3d-909c-c90fb1d0be64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:41.052096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fedc-2dfb-4c3d-909c-c90fb1d0be64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:41.052380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fedc-2dfb-4c3d-909c-c90fb1d0be64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:41.052660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546fedc-2dfb-4c3d-909c-c90fb1d0be64 admin admin] PATH_INFO: `/v3/projects/e9e044da83ce4ce287d14c2bf71417b8/users/3feb9ae821804e5e8d038b16303a2476/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:41.053077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776cbe2-454b-4e4d-b27e-18449717e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:41.053682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776cbe2-454b-4e4d-b27e-18449717e9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:41.053989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776cbe2-454b-4e4d-b27e-18449717e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:41.054269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776cbe2-454b-4e4d-b27e-18449717e9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:41.054907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776cbe2-454b-4e4d-b27e-18449717e9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:41.056532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776cbe2-454b-4e4d-b27e-18449717e9e8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:41.058026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:41.068784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 920/1865] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:41.069242 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546fedc-2dfb-4c3d-909c-c90fb1d0be64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e044da83ce4ce287d14c2bf71417b8, user_id=3feb9ae821804e5e8d038b16303a2476, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:41.072072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546fedc-2dfb-4c3d-909c-c90fb1d0be64 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:41.073672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776cbe2-454b-4e4d-b27e-18449717e9e8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:41.084138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776cbe2-454b-4e4d-b27e-18449717e9e8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:41.088797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab0f31-113d-4360-9d22-8e108d182dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:41.089442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab0f31-113d-4360-9d22-8e108d182dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:41.089724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab0f31-113d-4360-9d22-8e108d182dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:41.090029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab0f31-113d-4360-9d22-8e108d182dcf admin admin] PATH_INFO: `/v3/projects/07c4b472d8a84d33ae248c2dc1c4c010/users/4dd00a0a2d2f4a018782da0ca475e59b/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:41.091926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab0f31-113d-4360-9d22-8e108d182dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c4b472d8a84d33ae248c2dc1c4c010, user_id=4dd00a0a2d2f4a018782da0ca475e59b, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:41.094657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab0f31-113d-4360-9d22-8e108d182dcf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:41.105711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 921/1866] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:41 2022] PUT /identity/v3/projects/07c4b472d8a84d33ae248c2dc1c4c010/users/4dd00a0a2d2f4a018782da0ca475e59b/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:41.115791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8714410-3046-47b1-be8b-acb294f3da4a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:41.116112 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8714410-3046-47b1-be8b-acb294f3da4a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:41.116356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8714410-3046-47b1-be8b-acb294f3da4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:41.145302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e8714410-3046-47b1-be8b-acb294f3da4a None None] MFA Rules not processed for user `4dd00a0a2d2f4a018782da0ca475e59b`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:41.146533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e8714410-3046-47b1-be8b-acb294f3da4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:41.168300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 946/1867] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:40 2022] PUT /identity/v3/projects/e9e044da83ce4ce287d14c2bf71417b8/users/3feb9ae821804e5e8d038b16303a2476/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:41.172313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546fedc-2dfb-4c3d-909c-c90fb1d0be64 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:41.236885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 922/1868] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:41.245949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8714410-3046-47b1-be8b-acb294f3da4a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:41.247718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e8714410-3046-47b1-be8b-acb294f3da4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:41.250062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546fedc-2dfb-4c3d-909c-c90fb1d0be64 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:41.255531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fac14-337a-4fcc-8ef5-f3a890cb738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:41.256141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fac14-337a-4fcc-8ef5-f3a890cb738b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:41.256419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fac14-337a-4fcc-8ef5-f3a890cb738b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:41.256693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fac14-337a-4fcc-8ef5-f3a890cb738b admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:41.257332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fac14-337a-4fcc-8ef5-f3a890cb738b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:41.259064 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fac14-337a-4fcc-8ef5-f3a890cb738b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:41.269642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 947/1869] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:41.273760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fac14-337a-4fcc-8ef5-f3a890cb738b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:41.283782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fac14-337a-4fcc-8ef5-f3a890cb738b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:41.288464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a7c27-0d1d-43f3-833c-9dbc4d8d14a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:41.289096 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7c27-0d1d-43f3-833c-9dbc4d8d14a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:41.289369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7c27-0d1d-43f3-833c-9dbc4d8d14a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:41.289639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7c27-0d1d-43f3-833c-9dbc4d8d14a5 admin admin] PATH_INFO: `/v3/projects/e9e044da83ce4ce287d14c2bf71417b8/users/3feb9ae821804e5e8d038b16303a2476/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:41.291532 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a7c27-0d1d-43f3-833c-9dbc4d8d14a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e044da83ce4ce287d14c2bf71417b8, user_id=3feb9ae821804e5e8d038b16303a2476, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:41.294504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a7c27-0d1d-43f3-833c-9dbc4d8d14a5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:41.300163 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:41.305335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 948/1870] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:41 2022] PUT /identity/v3/projects/e9e044da83ce4ce287d14c2bf71417b8/users/3feb9ae821804e5e8d038b16303a2476/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:41.309347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8714410-3046-47b1-be8b-acb294f3da4a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:41.317576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad2cb-4d07-4747-9330-8aadb6ef851a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:41.317991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad2cb-4d07-4747-9330-8aadb6ef851a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:41.318474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efad2cb-4d07-4747-9330-8aadb6ef851a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:41.346764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2efad2cb-4d07-4747-9330-8aadb6ef851a None None] MFA Rules not processed for user `3feb9ae821804e5e8d038b16303a2476`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:41.347992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2efad2cb-4d07-4747-9330-8aadb6ef851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:41.364749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:41.365369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:41.365645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:41.365964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:41.367992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:41.408872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:41.424120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 949/1871] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:41.431099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:41.432530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:41.442942 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 923/1872] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:42.137583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efad2cb-4d07-4747-9330-8aadb6ef851a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:42.147622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efad2cb-4d07-4747-9330-8aadb6ef851a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:42.151154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5cd6f-b992-4719-b94e-6f4ad0e6a403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:42.151744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5cd6f-b992-4719-b94e-6f4ad0e6a403 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:42.152019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5cd6f-b992-4719-b94e-6f4ad0e6a403 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:42.152285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5cd6f-b992-4719-b94e-6f4ad0e6a403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:42.162401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5cd6f-b992-4719-b94e-6f4ad0e6a403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:42.163819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5cd6f-b992-4719-b94e-6f4ad0e6a403 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:42.173346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 950/1873] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:42.292983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:42.303125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8ef1f-f500-4ad0-b41f-ff95e4540e1f service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:42.306969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae5387-f017-4cb0-8b6f-d7fffcc48825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:42.308260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae5387-f017-4cb0-8b6f-d7fffcc48825 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:42.308260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae5387-f017-4cb0-8b6f-d7fffcc48825 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:42.308260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae5387-f017-4cb0-8b6f-d7fffcc48825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:42.366321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae5387-f017-4cb0-8b6f-d7fffcc48825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:42.368013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae5387-f017-4cb0-8b6f-d7fffcc48825 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:42.377609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 924/1874] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:42 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:42.580901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5cd6f-b992-4719-b94e-6f4ad0e6a403 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:42.589468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5cd6f-b992-4719-b94e-6f4ad0e6a403 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:42.593077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:42.593663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:42.593960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:42.594237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:42.595999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:42.664842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:42.666369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:42.676187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 951/1875] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:43.622356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae5387-f017-4cb0-8b6f-d7fffcc48825 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:43.638448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae5387-f017-4cb0-8b6f-d7fffcc48825 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:43.640356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a2b91-f890-4f81-946c-286cb2127a36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:43.640962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a2b91-f890-4f81-946c-286cb2127a36 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:43.641240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a2b91-f890-4f81-946c-286cb2127a36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:43.641508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a2b91-f890-4f81-946c-286cb2127a36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:43.707063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a2b91-f890-4f81-946c-286cb2127a36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:43.708968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a2b91-f890-4f81-946c-286cb2127a36 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:43.718464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 925/1876] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:45.215540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:45.225737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53531f99-e41c-4a92-ba11-07c89e26ca79 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:45.229774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:45.230544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:45.230836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:45.231240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:45.293097 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:45.294545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:45.298425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:45.304332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 952/1877] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:48.099345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a2b91-f890-4f81-946c-286cb2127a36 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:48.120684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:48.122293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:48.168189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a2b91-f890-4f81-946c-286cb2127a36 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:48.172858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:48.173503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:48.173796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:48.174107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] PATH_INFO: `/v3/users/87829e595f504195bd95fedb5648e31a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:48.175246 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87829e595f504195bd95fedb5648e31a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:48.177213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:48.201916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:48.203239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea2ada-00a3-4a50-9866-b0621ab3a8b6 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:48.210272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:48.210922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:48.211307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:48.211698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] PATH_INFO: `/v3/users/be20c635816b4f82b7448ab07e05adf3` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:48.212859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be20c635816b4f82b7448ab07e05adf3)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:48.214863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:48.383497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87829e595f504195bd95fedb5648e31a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.384285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be20c635816b4f82b7448ab07e05adf3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.396078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87829e595f504195bd95fedb5648e31a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.404244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be20c635816b4f82b7448ab07e05adf3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.404965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be20c635816b4f82b7448ab07e05adf3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.470963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87829e595f504195bd95fedb5648e31a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.471509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87829e595f504195bd95fedb5648e31a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.479389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-129df5eb-b4f7-41ee-a28a-ddef7b30f631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87829e595f504195bd95fedb5648e31a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.485201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 926/1878] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:48 2022] DELETE /identity/v3/users/87829e595f504195bd95fedb5648e31a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:48.489999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7f962-22fd-4be2-aaf2-4c812b2b08c6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:48.490312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7f962-22fd-4be2-aaf2-4c812b2b08c6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:48.490590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7f962-22fd-4be2-aaf2-4c812b2b08c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:48.510236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e8f7f962-22fd-4be2-aaf2-4c812b2b08c6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:48.515699 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e8f7f962-22fd-4be2-aaf2-4c812b2b08c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:48.541304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be20c635816b4f82b7448ab07e05adf3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.549784 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e6828a03-b2b8-4b13-b950-02906d864118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be20c635816b4f82b7448ab07e05adf3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:48.559280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 953/1879] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:48 2022] DELETE /identity/v3/users/be20c635816b4f82b7448ab07e05adf3 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:48.560724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949abfbb-e26c-4d18-a007-730f7fb318bf None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:48.561063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949abfbb-e26c-4d18-a007-730f7fb318bf None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:48.561420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949abfbb-e26c-4d18-a007-730f7fb318bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:48.582019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-949abfbb-e26c-4d18-a007-730f7fb318bf None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:48.583818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 927/1880] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:48.587207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-949abfbb-e26c-4d18-a007-730f7fb318bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:48.594257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7f962-22fd-4be2-aaf2-4c812b2b08c6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:48.595623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e8f7f962-22fd-4be2-aaf2-4c812b2b08c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:48.644737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 954/1881] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:48.655440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949abfbb-e26c-4d18-a007-730f7fb318bf None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:48.659338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7f962-22fd-4be2-aaf2-4c812b2b08c6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:48.677574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:48.729023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949abfbb-e26c-4d18-a007-730f7fb318bf None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:48.739545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:48.741170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:48.741170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:48.741170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:48.750225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:48.750839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:48.751113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:48.751386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:48.752411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:48.753845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:48.757995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:48.762898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:48.764359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:48.767082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 955/1882] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:48.774713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 928/1883] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:39:49.325951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:49.326719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:49.327879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:49.329019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:49.344954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a08b57-36be-407d-b3a4-fe27aa78ebcb service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:49.346406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae24f1-3d93-47e3-86f4-d8fb06a91541 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:49.353203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:49.353864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:49.354229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:49.354413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] PATH_INFO: `/v3/projects/000745d932fa489284f90333ec0fbbdb` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:49.355839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=000745d932fa489284f90333ec0fbbdb)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:49.358170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:49.359331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:49.359965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:49.361019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:49.361426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] PATH_INFO: `/v3/projects/5c324b4cd71a4a908615a43b451b54bd` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:49.362704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c324b4cd71a4a908615a43b451b54bd)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:49.365549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:49.461326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '000745d932fa489284f90333ec0fbbdb', 'request_id': 'req-27bf4494-95fc-4d72-b801-f12c77376ab0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.467000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '000745d932fa489284f90333ec0fbbdb', 'request_id': 'req-27bf4494-95fc-4d72-b801-f12c77376ab0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.474155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '000745d932fa489284f90333ec0fbbdb', 'request_id': 'req-27bf4494-95fc-4d72-b801-f12c77376ab0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.484434 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c324b4cd71a4a908615a43b451b54bd', 'request_id': 'req-79f4dfe7-47e0-485f-84d1-5d225f893ee6'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.489952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c324b4cd71a4a908615a43b451b54bd', 'request_id': 'req-79f4dfe7-47e0-485f-84d1-5d225f893ee6'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.496506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c324b4cd71a4a908615a43b451b54bd', 'request_id': 'req-79f4dfe7-47e0-485f-84d1-5d225f893ee6'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.555727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] The token cache is being invalidate because project 000745d932fa489284f90333ec0fbbdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:49.556057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.556635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.558153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 929/1884] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:49 2022] DELETE /identity/v3/projects/000745d932fa489284f90333ec0fbbdb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:49.561947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:49.563227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:49.614459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] The token cache is being invalidate because project 5c324b4cd71a4a908615a43b451b54bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:49.614879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.615606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.617650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 956/1885] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:49 2022] DELETE /identity/v3/projects/5c324b4cd71a4a908615a43b451b54bd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:49.623983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:49.625299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:49.627029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf4494-95fc-4d72-b801-f12c77376ab0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:49.684268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:49.685146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:49.685524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:49.685893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] PATH_INFO: `/v3/users/3feb9ae821804e5e8d038b16303a2476` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:49.687119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3feb9ae821804e5e8d038b16303a2476)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:49.689151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:49.693689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4dfe7-47e0-485f-84d1-5d225f893ee6 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:49.698427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:49.699055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:49.699333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:49.699634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] PATH_INFO: `/v3/users/bbf63eb44e2a4e22bbd087850cd4cb56` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:49.700765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbf63eb44e2a4e22bbd087850cd4cb56)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:49.702715 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:49.780619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:49.931023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:49.932242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3feb9ae821804e5e8d038b16303a2476'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.932568 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbf63eb44e2a4e22bbd087850cd4cb56'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.944139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3feb9ae821804e5e8d038b16303a2476'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.944566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbf63eb44e2a4e22bbd087850cd4cb56'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:49.945322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbf63eb44e2a4e22bbd087850cd4cb56'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.060808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3feb9ae821804e5e8d038b16303a2476'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.061419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3feb9ae821804e5e8d038b16303a2476'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.069452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3feb9ae821804e5e8d038b16303a2476'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.078392 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 930/1886] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:49 2022] DELETE /identity/v3/users/3feb9ae821804e5e8d038b16303a2476 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:50.152645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbf63eb44e2a4e22bbd087850cd4cb56'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.160536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbf63eb44e2a4e22bbd087850cd4cb56'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.166863 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 957/1887] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:49 2022] DELETE /identity/v3/users/bbf63eb44e2a4e22bbd087850cd4cb56 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:50.605479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:50.606963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:50.624908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:50.627415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:50.694753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5aec-8272-49e7-a047-a6df280648f6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:50.708174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535424e6-6e41-40b6-845c-e34913c688c6 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:50.713299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:50.713947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:50.714236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:50.714531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] PATH_INFO: `/v3/projects/1993472c5aea4e2f8edb21b4f1a54f1f` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:50.715671 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1993472c5aea4e2f8edb21b4f1a54f1f)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:50.717673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:50.753738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:50.754957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:50.755239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:50.755509 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] PATH_INFO: `/v3/projects/e9e044da83ce4ce287d14c2bf71417b8` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:50.756671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e044da83ce4ce287d14c2bf71417b8)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:50.758666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:50.822474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1993472c5aea4e2f8edb21b4f1a54f1f', 'request_id': 'req-1dda3eea-7867-4436-9efa-a149820f0f6b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.828234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1993472c5aea4e2f8edb21b4f1a54f1f', 'request_id': 'req-1dda3eea-7867-4436-9efa-a149820f0f6b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.835440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1993472c5aea4e2f8edb21b4f1a54f1f', 'request_id': 'req-1dda3eea-7867-4436-9efa-a149820f0f6b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.868501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e044da83ce4ce287d14c2bf71417b8', 'request_id': 'req-aad28917-08db-4de0-81f7-4f02980123db'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.870417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:50.874354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e044da83ce4ce287d14c2bf71417b8', 'request_id': 'req-aad28917-08db-4de0-81f7-4f02980123db'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.880703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e044da83ce4ce287d14c2bf71417b8', 'request_id': 'req-aad28917-08db-4de0-81f7-4f02980123db'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.912127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] The token cache is being invalidate because project 1993472c5aea4e2f8edb21b4f1a54f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:50.913682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.913682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.914694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 958/1888] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:50 2022] DELETE /identity/v3/projects/1993472c5aea4e2f8edb21b4f1a54f1f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:50.918546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:50.919893 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:50.971455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:50.979742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] The token cache is being invalidate because project e9e044da83ce4ce287d14c2bf71417b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:50.980033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.980671 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-aad28917-08db-4de0-81f7-4f02980123db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:50.982452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 931/1889] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:50 2022] DELETE /identity/v3/projects/e9e044da83ce4ce287d14c2bf71417b8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:50.984379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda3eea-7867-4436-9efa-a149820f0f6b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:50.987988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3fbff-dc0f-4d38-a1d5-a4144c042a36 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:50.988301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3fbff-dc0f-4d38-a1d5-a4144c042a36 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:50.988575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3fbff-dc0f-4d38-a1d5-a4144c042a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.009716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d5f3fbff-dc0f-4d38-a1d5-a4144c042a36 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:51.011007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d5f3fbff-dc0f-4d38-a1d5-a4144c042a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:51.041562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.042234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.042518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.042797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] PATH_INFO: `/v3/users/b37864f7329f47729a2b3fb5823c38d0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.043947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37864f7329f47729a2b3fb5823c38d0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.045865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:51.072009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 932/1890] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:51.079863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d555f-96ed-4057-9352-507d348c53b8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.080230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d555f-96ed-4057-9352-507d348c53b8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.080429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d555f-96ed-4057-9352-507d348c53b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.100295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f19d555f-96ed-4057-9352-507d348c53b8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:51.101497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f19d555f-96ed-4057-9352-507d348c53b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:51.109390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 933/1891] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:51.113529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d555f-96ed-4057-9352-507d348c53b8 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:51.123158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d555f-96ed-4057-9352-507d348c53b8 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:51.127769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.128338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.131860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.131860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.131860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:51.131860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.131860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:51.143030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 934/1892] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:39:51.148071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:51.157898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba9c95-1d7a-4316-8308-55c77d84e96e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:51.164105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba564b6-bf74-43c3-be07-81d1e13e54c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.164700 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba564b6-bf74-43c3-be07-81d1e13e54c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.164984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba564b6-bf74-43c3-be07-81d1e13e54c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.165351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba564b6-bf74-43c3-be07-81d1e13e54c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.166005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba564b6-bf74-43c3-be07-81d1e13e54c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.168087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba564b6-bf74-43c3-be07-81d1e13e54c2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:51.238921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37864f7329f47729a2b3fb5823c38d0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:51.253563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 935/1893] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:51 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:51.257460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba564b6-bf74-43c3-be07-81d1e13e54c2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:51.261874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37864f7329f47729a2b3fb5823c38d0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:51.262820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37864f7329f47729a2b3fb5823c38d0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:51.329119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba564b6-bf74-43c3-be07-81d1e13e54c2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:51.334412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c735d-f5e7-4b75-b1e7-568c2125a4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.337348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c735d-f5e7-4b75-b1e7-568c2125a4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.337348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c735d-f5e7-4b75-b1e7-568c2125a4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.337348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c735d-f5e7-4b75-b1e7-568c2125a4a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.337348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c735d-f5e7-4b75-b1e7-568c2125a4a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.338106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c735d-f5e7-4b75-b1e7-568c2125a4a5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:51.356966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37864f7329f47729a2b3fb5823c38d0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:51.364511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37864f7329f47729a2b3fb5823c38d0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:51.371060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 959/1894] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:50 2022] DELETE /identity/v3/users/b37864f7329f47729a2b3fb5823c38d0 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:51.439147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 936/1895] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:51 2022] POST /identity/v3/users => generated 284 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:51.442943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:51.444501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:51.514579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c264d1-b42e-46ad-ab44-17bce5aeed5e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:51.519196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe128b4-ae22-4aa8-883d-ac960d0a8cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.520514 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe128b4-ae22-4aa8-883d-ac960d0a8cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.520916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe128b4-ae22-4aa8-883d-ac960d0a8cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.521191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe128b4-ae22-4aa8-883d-ac960d0a8cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.521836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe128b4-ae22-4aa8-883d-ac960d0a8cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.523448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe128b4-ae22-4aa8-883d-ac960d0a8cc0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:51.533666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 960/1896] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:51.538474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c735d-f5e7-4b75-b1e7-568c2125a4a5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:51.548684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c735d-f5e7-4b75-b1e7-568c2125a4a5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:51.554060 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ea8cf-09f4-42b4-96c3-a6ccb0ef0d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.554797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea8cf-09f4-42b4-96c3-a6ccb0ef0d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.555058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea8cf-09f4-42b4-96c3-a6ccb0ef0d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.555329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ea8cf-09f4-42b4-96c3-a6ccb0ef0d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.555993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea8cf-09f4-42b4-96c3-a6ccb0ef0d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.557603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ea8cf-09f4-42b4-96c3-a6ccb0ef0d8f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:51.572764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 937/1897] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:51.574363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe128b4-ae22-4aa8-883d-ac960d0a8cc0 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:51.591800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe128b4-ae22-4aa8-883d-ac960d0a8cc0 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:51.606781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.608785 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.614941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.614941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] PATH_INFO: `/v3/projects/683c6772e367473396006f9f21ddd167/users/ced9a74b75ea4a48877517e08536f1eb/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.631052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:51.637867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683c6772e367473396006f9f21ddd167, user_id=ced9a74b75ea4a48877517e08536f1eb, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.640087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:51.698546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 961/1898] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:51 2022] PUT /identity/v3/projects/683c6772e367473396006f9f21ddd167/users/ced9a74b75ea4a48877517e08536f1eb/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:51.709574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb933df-8a97-40b5-a4d8-a65ee1d8bd3b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.710148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb933df-8a97-40b5-a4d8-a65ee1d8bd3b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.710148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb933df-8a97-40b5-a4d8-a65ee1d8bd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.775856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-feb933df-8a97-40b5-a4d8-a65ee1d8bd3b None None] MFA Rules not processed for user `ced9a74b75ea4a48877517e08536f1eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:51.777105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-feb933df-8a97-40b5-a4d8-a65ee1d8bd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:51.787734 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:51.852455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:51.855645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:51.881017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 938/1899] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:51.885420 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb933df-8a97-40b5-a4d8-a65ee1d8bd3b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:51.962377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb933df-8a97-40b5-a4d8-a65ee1d8bd3b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:51.967463 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf05b-c986-4ff8-90b0-34ebd828ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.968080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf05b-c986-4ff8-90b0-34ebd828ded6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.968359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf05b-c986-4ff8-90b0-34ebd828ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.968630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf05b-c986-4ff8-90b0-34ebd828ded6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.969266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaf05b-c986-4ff8-90b0-34ebd828ded6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.970910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cf745-f875-49e4-8337-c058b1d16a73 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:51.971361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaf05b-c986-4ff8-90b0-34ebd828ded6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:51.976199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:51.978480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:51.978480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:51.978480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] PATH_INFO: `/v3/projects/a6410a34e84a475c9583f2f399baf2c2` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:51.978744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6410a34e84a475c9583f2f399baf2c2)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:51.980394 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.077706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 939/1900] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:51 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:39:52.079524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf05b-c986-4ff8-90b0-34ebd828ded6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.141375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf05b-c986-4ff8-90b0-34ebd828ded6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.148361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6410a34e84a475c9583f2f399baf2c2', 'request_id': 'req-cd28f02b-695d-4a40-b72c-1f6556c175a2'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:52.155097 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6410a34e84a475c9583f2f399baf2c2', 'request_id': 'req-cd28f02b-695d-4a40-b72c-1f6556c175a2'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:52.162752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6410a34e84a475c9583f2f399baf2c2', 'request_id': 'req-cd28f02b-695d-4a40-b72c-1f6556c175a2'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:52.180257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:52.198450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.199110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.199387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.199685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.200321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.202284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.226815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] The token cache is being invalidate because project a6410a34e84a475c9583f2f399baf2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:52.227102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:52.227727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cd28f02b-695d-4a40-b72c-1f6556c175a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:52.229261 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 962/1901] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:51 2022] DELETE /identity/v3/projects/a6410a34e84a475c9583f2f399baf2c2 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:52.234162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ffc6-4a77-42b5-ae3c-a63a9a292fe6 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.234440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ffc6-4a77-42b5-ae3c-a63a9a292fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.234707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ffc6-4a77-42b5-ae3c-a63a9a292fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.254934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-dbf9ffc6-4a77-42b5-ae3c-a63a9a292fe6 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:52.256125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-dbf9ffc6-4a77-42b5-ae3c-a63a9a292fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:52.264095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 963/1902] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:52.271752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f4f9b-b45b-40b7-802c-3eb47192f6e3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.272034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f4f9b-b45b-40b7-802c-3eb47192f6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.272311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f4f9b-b45b-40b7-802c-3eb47192f6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.281412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 940/1903] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:52 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:52.282605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.287421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:52.294089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-bc1f4f9b-b45b-40b7-802c-3eb47192f6e3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:52.295539 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bc1f4f9b-b45b-40b7-802c-3eb47192f6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:52.299621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb011a7-9b88-4f06-ab43-72b4532470f4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.304889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d655a-f6e3-44ae-9dd7-68e4396e21fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.305489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d655a-f6e3-44ae-9dd7-68e4396e21fa admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.305764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d655a-f6e3-44ae-9dd7-68e4396e21fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.306065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d655a-f6e3-44ae-9dd7-68e4396e21fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.306703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d655a-f6e3-44ae-9dd7-68e4396e21fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.308270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d655a-f6e3-44ae-9dd7-68e4396e21fa admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.309478 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 964/1904] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:52.310218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f4f9b-b45b-40b7-802c-3eb47192f6e3 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.319992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 941/1905] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:52.322718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f4f9b-b45b-40b7-802c-3eb47192f6e3 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.324114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d655a-f6e3-44ae-9dd7-68e4396e21fa admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.328695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.329300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.329581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.329846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.330523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:52.330825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.332447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.336082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d655a-f6e3-44ae-9dd7-68e4396e21fa admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.341772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d88b7-4a69-493a-8cab-d6b27b75ebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.342464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d88b7-4a69-493a-8cab-d6b27b75ebc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.342743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d88b7-4a69-493a-8cab-d6b27b75ebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.343014 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d88b7-4a69-493a-8cab-d6b27b75ebc6 admin admin] PATH_INFO: `/v3/projects/42cc467a8b2a418999855bd196aa5f03/users/96af3a0f0e814576a794b6e806118a6a/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.344847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 965/1906] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:52.349254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.358144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d88b7-4a69-493a-8cab-d6b27b75ebc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cc467a8b2a418999855bd196aa5f03, user_id=96af3a0f0e814576a794b6e806118a6a, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.358830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325ed54-6d84-456c-a119-23994c1dccb7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.360786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d88b7-4a69-493a-8cab-d6b27b75ebc6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.363378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9a7e-5861-4c0f-80e6-1af5cc926001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.363994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9a7e-5861-4c0f-80e6-1af5cc926001 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.364269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9a7e-5861-4c0f-80e6-1af5cc926001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.364536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee9a7e-5861-4c0f-80e6-1af5cc926001 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.365146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee9a7e-5861-4c0f-80e6-1af5cc926001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.367047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee9a7e-5861-4c0f-80e6-1af5cc926001 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.426989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 942/1907] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:52 2022] PUT /identity/v3/projects/42cc467a8b2a418999855bd196aa5f03/users/96af3a0f0e814576a794b6e806118a6a/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:52.430690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d88b7-4a69-493a-8cab-d6b27b75ebc6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.461586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 966/1908] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:52 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:52.462592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9a7e-5861-4c0f-80e6-1af5cc926001 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.506667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:52.509626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d88b7-4a69-493a-8cab-d6b27b75ebc6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.535777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee9a7e-5861-4c0f-80e6-1af5cc926001 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.541005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbe4d5-dddc-4e93-9499-dfdf1f02ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.541602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbe4d5-dddc-4e93-9499-dfdf1f02ad3b admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.541883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbe4d5-dddc-4e93-9499-dfdf1f02ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.542178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbe4d5-dddc-4e93-9499-dfdf1f02ad3b admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.542801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbe4d5-dddc-4e93-9499-dfdf1f02ad3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.544708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbe4d5-dddc-4e93-9499-dfdf1f02ad3b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.575130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7094919-ead7-458c-a67b-c483792a44f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.575130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7094919-ead7-458c-a67b-c483792a44f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.575130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7094919-ead7-458c-a67b-c483792a44f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.575130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7094919-ead7-458c-a67b-c483792a44f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.575469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7094919-ead7-458c-a67b-c483792a44f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.575875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7094919-ead7-458c-a67b-c483792a44f7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.589908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 943/1909] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:52.590921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7094919-ead7-458c-a67b-c483792a44f7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.601369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7094919-ead7-458c-a67b-c483792a44f7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.606458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9460285-7bd9-4b35-86a3-379196717efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.607109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460285-7bd9-4b35-86a3-379196717efe admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.607601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460285-7bd9-4b35-86a3-379196717efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.607881 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460285-7bd9-4b35-86a3-379196717efe admin admin] PATH_INFO: `/v3/projects/42cc467a8b2a418999855bd196aa5f03/users/96af3a0f0e814576a794b6e806118a6a/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.609853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9460285-7bd9-4b35-86a3-379196717efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cc467a8b2a418999855bd196aa5f03, user_id=96af3a0f0e814576a794b6e806118a6a, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.612594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9460285-7bd9-4b35-86a3-379196717efe admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.622916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 967/1910] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:52 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:52.624596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 944/1911] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:52 2022] PUT /identity/v3/projects/42cc467a8b2a418999855bd196aa5f03/users/96af3a0f0e814576a794b6e806118a6a/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:52.627588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbe4d5-dddc-4e93-9499-dfdf1f02ad3b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.636276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe55e1-07e0-41e9-9d5e-011145960e34 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.636602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe55e1-07e0-41e9-9d5e-011145960e34 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.636805 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe55e1-07e0-41e9-9d5e-011145960e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.638918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbe4d5-dddc-4e93-9499-dfdf1f02ad3b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.671415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5ffe55e1-07e0-41e9-9d5e-011145960e34 None None] MFA Rules not processed for user `96af3a0f0e814576a794b6e806118a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:52.672715 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5ffe55e1-07e0-41e9-9d5e-011145960e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:52.686465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:52.702045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975ef7b-6340-49bf-9c5e-265549ef4664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.702887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975ef7b-6340-49bf-9c5e-265549ef4664 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.703173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975ef7b-6340-49bf-9c5e-265549ef4664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.703446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975ef7b-6340-49bf-9c5e-265549ef4664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.704125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975ef7b-6340-49bf-9c5e-265549ef4664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.709914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975ef7b-6340-49bf-9c5e-265549ef4664 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.719723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 968/1912] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:52.720905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975ef7b-6340-49bf-9c5e-265549ef4664 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.731164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975ef7b-6340-49bf-9c5e-265549ef4664 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.735808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.736436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.736719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.736992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.738023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.739781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.756736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 945/1913] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:39:52.757241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 969/1914] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:39:52.759636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe55e1-07e0-41e9-9d5e-011145960e34 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.767215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:52.769642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:52.771007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe55e1-07e0-41e9-9d5e-011145960e34 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.775848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.776513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.776794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.777069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] PATH_INFO: `/v3/projects/b15d78e196664c1aa41be6c5afe4238b/users/b0a7f349d8ba4bea9a609e7cf881b1b9/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.791682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15d78e196664c1aa41be6c5afe4238b, user_id=b0a7f349d8ba4bea9a609e7cf881b1b9, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.794465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.827761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5e13-3e27-43b9-a4c5-d92fb50055cf admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:52.831436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70563d9-4141-4aff-bf6d-e7421543de27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:52.832029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70563d9-4141-4aff-bf6d-e7421543de27 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.832309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70563d9-4141-4aff-bf6d-e7421543de27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.832705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70563d9-4141-4aff-bf6d-e7421543de27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.842145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70563d9-4141-4aff-bf6d-e7421543de27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:52.843856 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70563d9-4141-4aff-bf6d-e7421543de27 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:52.854284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 970/1915] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:52.874024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 946/1916] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:52 2022] PUT /identity/v3/projects/b15d78e196664c1aa41be6c5afe4238b/users/b0a7f349d8ba4bea9a609e7cf881b1b9/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:52.879101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3215-1bc7-4cb6-8aca-9a61f3609125 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:52.880046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3215-1bc7-4cb6-8aca-9a61f3609125 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:52.880353 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3215-1bc7-4cb6-8aca-9a61f3609125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:52.900773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2fdf3215-1bc7-4cb6-8aca-9a61f3609125 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:52.902206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2fdf3215-1bc7-4cb6-8aca-9a61f3609125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:52.958821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:52.980189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 971/1917] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:52.991296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:53.003269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:53.089416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42221d04-5fed-433f-aece-fbf798159643 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:53.093127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:53.093797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:53.094042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:53.094319 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:53.103711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:53.105133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:53.115129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 947/1918] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:53.182681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf3215-1bc7-4cb6-8aca-9a61f3609125 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:53.184207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2fdf3215-1bc7-4cb6-8aca-9a61f3609125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:53.246495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf3215-1bc7-4cb6-8aca-9a61f3609125 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:53.250366 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e3c93-f1e1-4a60-bce5-d93c6248ebe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:53.250946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e3c93-f1e1-4a60-bce5-d93c6248ebe5 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:53.251228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e3c93-f1e1-4a60-bce5-d93c6248ebe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:53.251493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e3c93-f1e1-4a60-bce5-d93c6248ebe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:53.253547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-df4e3c93-f1e1-4a60-bce5-d93c6248ebe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:53.316680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e3c93-f1e1-4a60-bce5-d93c6248ebe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:53.318142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e3c93-f1e1-4a60-bce5-d93c6248ebe5 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:53.327218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 972/1919] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:54.494967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:54.497262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:54.513253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290bd10-0796-46b0-87ec-42341b8a5e51 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:54.518575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:54.519430 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:54.519907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:54.520180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] PATH_INFO: `/v3/users/fa54a46f4fe14ca39c20e10b258cf8f7` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:54.521309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa54a46f4fe14ca39c20e10b258cf8f7)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:54.523291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:54.823781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa54a46f4fe14ca39c20e10b258cf8f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:54.862791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa54a46f4fe14ca39c20e10b258cf8f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:54.947243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa54a46f4fe14ca39c20e10b258cf8f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:54.949955 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa54a46f4fe14ca39c20e10b258cf8f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:54.961236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa54a46f4fe14ca39c20e10b258cf8f7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:54.971660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:55.006663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 948/1920] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:54 2022] DELETE /identity/v3/users/fa54a46f4fe14ca39c20e10b258cf8f7 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:55.011558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a26255-8c58-4966-a3c4-81e543e83d58 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:55.012005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a26255-8c58-4966-a3c4-81e543e83d58 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:55.012403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a26255-8c58-4966-a3c4-81e543e83d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:55.040733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-46a26255-8c58-4966-a3c4-81e543e83d58 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:55.042650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-46a26255-8c58-4966-a3c4-81e543e83d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:55.203431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 973/1921] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:55.215334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:55.217316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:55.286947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b493d2-32e5-45ee-8202-3226ebfa0a0a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:55.290560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b500774-ebd7-4877-99c1-52f916edfcf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:55.291221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500774-ebd7-4877-99c1-52f916edfcf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:55.291449 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500774-ebd7-4877-99c1-52f916edfcf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:55.291716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500774-ebd7-4877-99c1-52f916edfcf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:55.303183 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b500774-ebd7-4877-99c1-52f916edfcf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:55.304670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b500774-ebd7-4877-99c1-52f916edfcf0 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:55.315355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 949/1922] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:55.939215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a26255-8c58-4966-a3c4-81e543e83d58 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:55.940763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-46a26255-8c58-4966-a3c4-81e543e83d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:55.966105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a26255-8c58-4966-a3c4-81e543e83d58 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:55.974710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:55.976245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:55.976531 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:55.976804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] PATH_INFO: `/v3/projects/9270d8acba3e49028f0e3da555172c8f` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:55.977997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9270d8acba3e49028f0e3da555172c8f)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:55.987143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:56.161840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:56.168229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9270d8acba3e49028f0e3da555172c8f', 'request_id': 'req-58da4a5d-c6c1-49fc-9371-661589bd649b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.174546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9270d8acba3e49028f0e3da555172c8f', 'request_id': 'req-58da4a5d-c6c1-49fc-9371-661589bd649b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.182156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9270d8acba3e49028f0e3da555172c8f', 'request_id': 'req-58da4a5d-c6c1-49fc-9371-661589bd649b'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.323129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] The token cache is being invalidate because project 9270d8acba3e49028f0e3da555172c8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:56.324176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.324888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.326847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 974/1923] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:55 2022] DELETE /identity/v3/projects/9270d8acba3e49028f0e3da555172c8f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:56.330765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b500774-ebd7-4877-99c1-52f916edfcf0 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:56.342005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b500774-ebd7-4877-99c1-52f916edfcf0 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:56.424122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:56.424982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:56.425360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:56.425735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] PATH_INFO: `/v3/users/8a4480b703944a349a1ad100b3acc84c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:56.427185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a4480b703944a349a1ad100b3acc84c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:56.429540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:56.674281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a4480b703944a349a1ad100b3acc84c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.692531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a4480b703944a349a1ad100b3acc84c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.765105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a4480b703944a349a1ad100b3acc84c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.771399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a4480b703944a349a1ad100b3acc84c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.791905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a4480b703944a349a1ad100b3acc84c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:56.821752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 950/1924] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:56 2022] DELETE /identity/v3/users/8a4480b703944a349a1ad100b3acc84c => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:57.524148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:57.529445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:57.547485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da4a5d-c6c1-49fc-9371-661589bd649b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:57.616077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:57.616721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:57.617002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:57.617528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] PATH_INFO: `/v3/projects/9ee0e936b25f4c93afcd34e34a75c206` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:57.618688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee0e936b25f4c93afcd34e34a75c206)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:57.620609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:57.781467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee0e936b25f4c93afcd34e34a75c206', 'request_id': 'req-d950cdde-b388-49b6-ab44-beea2bdb33fd'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:57.788675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee0e936b25f4c93afcd34e34a75c206', 'request_id': 'req-d950cdde-b388-49b6-ab44-beea2bdb33fd'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:57.796898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee0e936b25f4c93afcd34e34a75c206', 'request_id': 'req-d950cdde-b388-49b6-ab44-beea2bdb33fd'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:57.925193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] The token cache is being invalidate because project 9ee0e936b25f4c93afcd34e34a75c206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:57.925193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:57.925193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:57.925193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 975/1925] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:57 2022] DELETE /identity/v3/projects/9ee0e936b25f4c93afcd34e34a75c206 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:57.928875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:57.930799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:57.952900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b245d-1524-43e7-a5d1-5572982ab74f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:57.993386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:58.039645 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:58.040550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:58.040830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:58.041098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] PATH_INFO: `/v3/users/4dd00a0a2d2f4a018782da0ca475e59b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:58.042414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd00a0a2d2f4a018782da0ca475e59b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:58.044462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:58.299024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd00a0a2d2f4a018782da0ca475e59b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:58.324078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd00a0a2d2f4a018782da0ca475e59b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:58.455307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd00a0a2d2f4a018782da0ca475e59b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:58.456144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd00a0a2d2f4a018782da0ca475e59b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:58.467554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-8d8126c1-7ab4-4c59-8152-1dcabb1f78a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd00a0a2d2f4a018782da0ca475e59b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:58.477998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 951/1926] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:57 2022] DELETE /identity/v3/users/4dd00a0a2d2f4a018782da0ca475e59b => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:58.960853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:58.963080 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:58.980202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950cdde-b388-49b6-ab44-beea2bdb33fd admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:59.172732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:59.179017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:59.179561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:59.180281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] PATH_INFO: `/v3/projects/07c4b472d8a84d33ae248c2dc1c4c010` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:59.181773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c4b472d8a84d33ae248c2dc1c4c010)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:59.185361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:59.187350 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:59.319841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c4b472d8a84d33ae248c2dc1c4c010', 'request_id': 'req-2abec5c0-193f-44a4-97b3-8e08dbef656e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:59.342409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c4b472d8a84d33ae248c2dc1c4c010', 'request_id': 'req-2abec5c0-193f-44a4-97b3-8e08dbef656e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:59.349677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c4b472d8a84d33ae248c2dc1c4c010', 'request_id': 'req-2abec5c0-193f-44a4-97b3-8e08dbef656e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:59.439129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] The token cache is being invalidate because project 07c4b472d8a84d33ae248c2dc1c4c010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:59.440116 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:59.443475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2abec5c0-193f-44a4-97b3-8e08dbef656e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:59.445562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 976/1927] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:58 2022] DELETE /identity/v3/projects/07c4b472d8a84d33ae248c2dc1c4c010 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:59.452182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d721e27-100f-40fe-88ce-2c248aa9005f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:59.452630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d721e27-100f-40fe-88ce-2c248aa9005f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:59.452798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d721e27-100f-40fe-88ce-2c248aa9005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:59.478953 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6d721e27-100f-40fe-88ce-2c248aa9005f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:59.481175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6d721e27-100f-40fe-88ce-2c248aa9005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:59.565591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 952/1928] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:59.573999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d656a-9a79-4c55-9675-1f3dcf6defec None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:59.574323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d656a-9a79-4c55-9675-1f3dcf6defec None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:59.574615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d656a-9a79-4c55-9675-1f3dcf6defec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:59.604520 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-af4d656a-9a79-4c55-9675-1f3dcf6defec None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:59.605883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-af4d656a-9a79-4c55-9675-1f3dcf6defec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:59.617121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 977/1929] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:59.622102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d721e27-100f-40fe-88ce-2c248aa9005f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:59.633462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d721e27-100f-40fe-88ce-2c248aa9005f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:59.639015 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:59.639776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:59.640113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:59.640457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:59.641230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:59.641586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:59.643375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:59.666932 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 953/1930] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:59.671640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d656a-9a79-4c55-9675-1f3dcf6defec None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:59.683678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d656a-9a79-4c55-9675-1f3dcf6defec None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:59.689845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580db5c5-d4c5-4de1-8eb2-bee49132806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:59.690559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db5c5-d4c5-4de1-8eb2-bee49132806e admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:59.690945 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db5c5-d4c5-4de1-8eb2-bee49132806e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:59.691268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db5c5-d4c5-4de1-8eb2-bee49132806e admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:59.692472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580db5c5-d4c5-4de1-8eb2-bee49132806e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:59.694570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580db5c5-d4c5-4de1-8eb2-bee49132806e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:59.795793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 978/1931] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:59 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:59.799784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:59.812017 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dd69a-d0a5-4681-8079-3857ee9d40fe admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:59.844162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:59.887904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486909de-4739-47a9-af24-29ed8eabce2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:59.888683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486909de-4739-47a9-af24-29ed8eabce2c admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:59.889174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486909de-4739-47a9-af24-29ed8eabce2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:59.889384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486909de-4739-47a9-af24-29ed8eabce2c admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:59.890158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486909de-4739-47a9-af24-29ed8eabce2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:59.892384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486909de-4739-47a9-af24-29ed8eabce2c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.005410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 954/1932] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:59 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:00.009624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580db5c5-d4c5-4de1-8eb2-bee49132806e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:00.027236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580db5c5-d4c5-4de1-8eb2-bee49132806e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:00.032745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b153449-9d34-4fa9-b007-5a81b66ca9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:00.033374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b153449-9d34-4fa9-b007-5a81b66ca9ce admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.033652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b153449-9d34-4fa9-b007-5a81b66ca9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.034063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b153449-9d34-4fa9-b007-5a81b66ca9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.034758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b153449-9d34-4fa9-b007-5a81b66ca9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:00.036892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b153449-9d34-4fa9-b007-5a81b66ca9ce admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.052304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 979/1933] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:00.052930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486909de-4739-47a9-af24-29ed8eabce2c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:00.081223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486909de-4739-47a9-af24-29ed8eabce2c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:00.089323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f025d7-8563-4d50-962c-4e8edc3c9154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:00.090002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f025d7-8563-4d50-962c-4e8edc3c9154 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.090261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f025d7-8563-4d50-962c-4e8edc3c9154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.096993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f025d7-8563-4d50-962c-4e8edc3c9154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.096993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f025d7-8563-4d50-962c-4e8edc3c9154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:00.096993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f025d7-8563-4d50-962c-4e8edc3c9154 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.110363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 955/1934] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:40:00.111519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b153449-9d34-4fa9-b007-5a81b66ca9ce admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:00.126127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b153449-9d34-4fa9-b007-5a81b66ca9ce admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:00.134829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f18bc7-a00b-4419-9e20-d7ac0162b037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:00.135655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18bc7-a00b-4419-9e20-d7ac0162b037 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.136002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18bc7-a00b-4419-9e20-d7ac0162b037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.136345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18bc7-a00b-4419-9e20-d7ac0162b037 admin admin] PATH_INFO: `/v3/projects/7b2052c4ed6041ce9d5fc06aa845ed6a/users/7df96c7bf04c4d4f91a5e4903159d5d1/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.165802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f18bc7-a00b-4419-9e20-d7ac0162b037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2052c4ed6041ce9d5fc06aa845ed6a, user_id=7df96c7bf04c4d4f91a5e4903159d5d1, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:00.169106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f18bc7-a00b-4419-9e20-d7ac0162b037 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.218327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a0a4a-bd24-465a-a628-1977ececfa86 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.218806 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a0a4a-bd24-465a-a628-1977ececfa86 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.219236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a0a4a-bd24-465a-a628-1977ececfa86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.270021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-104a0a4a-bd24-465a-a628-1977ececfa86 None None] MFA Rules not processed for user `b0a7f349d8ba4bea9a609e7cf881b1b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:00.270021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-104a0a4a-bd24-465a-a628-1977ececfa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:00.278070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 980/1935] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:40:00 2022] PUT /identity/v3/projects/7b2052c4ed6041ce9d5fc06aa845ed6a/users/7df96c7bf04c4d4f91a5e4903159d5d1/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:40:00.288413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f58e-892a-4dfd-8ba4-bed0047eaab9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.288699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f58e-892a-4dfd-8ba4-bed0047eaab9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.288975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7f58e-892a-4dfd-8ba4-bed0047eaab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.322393 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d0c7f58e-892a-4dfd-8ba4-bed0047eaab9 None None] MFA Rules not processed for user `7df96c7bf04c4d4f91a5e4903159d5d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:00.324479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d0c7f58e-892a-4dfd-8ba4-bed0047eaab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:00.385376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 956/1936] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:00.386473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a0a4a-bd24-465a-a628-1977ececfa86 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:00.388847 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-104a0a4a-bd24-465a-a628-1977ececfa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:00.453017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 981/1937] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:00.465416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7f58e-892a-4dfd-8ba4-bed0047eaab9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:00.466904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d0c7f58e-892a-4dfd-8ba4-bed0047eaab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:00.496272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a0a4a-bd24-465a-a628-1977ececfa86 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:00.497640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:00.505470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416e3af-38b4-4888-ac95-0dfd05064c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:00.506139 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416e3af-38b4-4888-ac95-0dfd05064c1c admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.506455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416e3af-38b4-4888-ac95-0dfd05064c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.506676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416e3af-38b4-4888-ac95-0dfd05064c1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.507770 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416e3af-38b4-4888-ac95-0dfd05064c1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:00.510396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416e3af-38b4-4888-ac95-0dfd05064c1c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.544603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7f58e-892a-4dfd-8ba4-bed0047eaab9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:00.548657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed088-2a41-45f7-860b-3858146bb260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:00.549273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed088-2a41-45f7-860b-3858146bb260 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.549555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed088-2a41-45f7-860b-3858146bb260 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.549826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed088-2a41-45f7-860b-3858146bb260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.559123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed088-2a41-45f7-860b-3858146bb260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:00.560553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed088-2a41-45f7-860b-3858146bb260 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.570004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 982/1938] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:00.616002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 957/1939] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:40:00 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:00.620519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed088-2a41-45f7-860b-3858146bb260 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:00.633390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed088-2a41-45f7-860b-3858146bb260 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:00.708870 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53651a3e-e79e-4195-98e5-88a6cd553cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:00.709524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53651a3e-e79e-4195-98e5-88a6cd553cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.709808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53651a3e-e79e-4195-98e5-88a6cd553cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.710116 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53651a3e-e79e-4195-98e5-88a6cd553cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.710754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53651a3e-e79e-4195-98e5-88a6cd553cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:00.712674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53651a3e-e79e-4195-98e5-88a6cd553cb4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.828684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 983/1940] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:40:00 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:00.831905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416e3af-38b4-4888-ac95-0dfd05064c1c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:00.843160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416e3af-38b4-4888-ac95-0dfd05064c1c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:00.847745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b84c2-36f0-402d-a87d-6780a10258a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:00.848923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b84c2-36f0-402d-a87d-6780a10258a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.849222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b84c2-36f0-402d-a87d-6780a10258a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.849499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b84c2-36f0-402d-a87d-6780a10258a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.850218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b84c2-36f0-402d-a87d-6780a10258a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:00.851842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b84c2-36f0-402d-a87d-6780a10258a0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.863391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 958/1941] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:00.868630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53651a3e-e79e-4195-98e5-88a6cd553cb4 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:00.878892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53651a3e-e79e-4195-98e5-88a6cd553cb4 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:00.884269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd16b81-a696-4b86-a0ba-151acbbf0b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:00.884943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd16b81-a696-4b86-a0ba-151acbbf0b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:00.885216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd16b81-a696-4b86-a0ba-151acbbf0b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:00.885714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd16b81-a696-4b86-a0ba-151acbbf0b06 admin admin] PATH_INFO: `/v3/projects/26d5ba2a9a0b401ea29a47f5b0e047e6/users/be7509d5cdca46be91f7259e320efd5d/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:00.902574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd16b81-a696-4b86-a0ba-151acbbf0b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d5ba2a9a0b401ea29a47f5b0e047e6, user_id=be7509d5cdca46be91f7259e320efd5d, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:00.905576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd16b81-a696-4b86-a0ba-151acbbf0b06 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:00.998643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 984/1942] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:40:00 2022] PUT /identity/v3/projects/26d5ba2a9a0b401ea29a47f5b0e047e6/users/be7509d5cdca46be91f7259e320efd5d/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:01.002695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b84c2-36f0-402d-a87d-6780a10258a0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:01.012554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b84c2-36f0-402d-a87d-6780a10258a0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:01.023326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:01.094373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:01.095201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:01.095454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:01.095730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:01.096567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:01.099729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:01.109465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 959/1943] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:01.113859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd16b81-a696-4b86-a0ba-151acbbf0b06 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:01.125560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd16b81-a696-4b86-a0ba-151acbbf0b06 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:01.131404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:01.131719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:01.132003 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:01.132280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] PATH_INFO: `/v3/projects/26d5ba2a9a0b401ea29a47f5b0e047e6/users/be7509d5cdca46be91f7259e320efd5d/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:01.134265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d5ba2a9a0b401ea29a47f5b0e047e6, user_id=be7509d5cdca46be91f7259e320efd5d, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:01.136923 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:01.148708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 985/1944] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:40:01 2022] PUT /identity/v3/projects/26d5ba2a9a0b401ea29a47f5b0e047e6/users/be7509d5cdca46be91f7259e320efd5d/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:01.658103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:01.666097 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:01.689877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc3645-2438-43ef-a400-31f2c25c0513 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:01.780760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:01.781695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:01.782349 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:01.782770 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] PATH_INFO: `/v3/users/34b3f7900b844581989376731d246407` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:01.784088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b3f7900b844581989376731d246407)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:01.787428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:01.826955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:01.829249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:01.846506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05697c07-49e2-4ed0-a439-ca82d9deb697 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:01.852623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:01.853499 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:01.853925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:01.854423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] PATH_INFO: `/v3/users/ced9a74b75ea4a48877517e08536f1eb` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:01.855772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced9a74b75ea4a48877517e08536f1eb)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:01.857969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:01.960062 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b3f7900b844581989376731d246407'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:01.976352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b3f7900b844581989376731d246407'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.061589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b3f7900b844581989376731d246407'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.062263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b3f7900b844581989376731d246407'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.071691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-46a0b913-6283-468f-827a-824231a35da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b3f7900b844581989376731d246407'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.080065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 960/1945] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:40:01 2022] DELETE /identity/v3/users/34b3f7900b844581989376731d246407 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:02.086318 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40797fb0-0de9-476a-b3db-82024fbed49e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:02.086608 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40797fb0-0de9-476a-b3db-82024fbed49e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:02.087261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40797fb0-0de9-476a-b3db-82024fbed49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:02.108571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced9a74b75ea4a48877517e08536f1eb'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.111884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-40797fb0-0de9-476a-b3db-82024fbed49e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:02.113473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-40797fb0-0de9-476a-b3db-82024fbed49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:02.126362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced9a74b75ea4a48877517e08536f1eb'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.127595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced9a74b75ea4a48877517e08536f1eb'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.130574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:02.146307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:02.204017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced9a74b75ea4a48877517e08536f1eb'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.213843 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e453e5cf-b9f6-47e6-8497-8e562833c5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced9a74b75ea4a48877517e08536f1eb'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:02.214243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 961/1946] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:02.232677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40797fb0-0de9-476a-b3db-82024fbed49e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:02.232872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 986/1947] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:40:01 2022] DELETE /identity/v3/users/ced9a74b75ea4a48877517e08536f1eb => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:02.233454 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-40797fb0-0de9-476a-b3db-82024fbed49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:02.234402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105709-318d-48f3-9d94-9cd5cb7440dd None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:02.237989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105709-318d-48f3-9d94-9cd5cb7440dd None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:02.237989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5105709-318d-48f3-9d94-9cd5cb7440dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:02.260999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b5105709-318d-48f3-9d94-9cd5cb7440dd None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:02.262897 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b5105709-318d-48f3-9d94-9cd5cb7440dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:02.342938 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40797fb0-0de9-476a-b3db-82024fbed49e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:02.375777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 987/1948] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:02.388954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105709-318d-48f3-9d94-9cd5cb7440dd None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:02.394850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b5105709-318d-48f3-9d94-9cd5cb7440dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:02.432732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:02.433722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:02.434031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:02.437654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:02.449275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:02.451235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:02.468185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 962/1949] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:02.487407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5105709-318d-48f3-9d94-9cd5cb7440dd None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:02.493161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:02.493161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:02.493161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:02.493161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:02.506103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:02.506103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:02.516191 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 988/1950] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:03.248707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:03.250823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:03.269864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:03.271486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:03.288707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056888b-94bf-46a6-a124-06a03c8e2e29 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:03.294138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:03.294677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910e967-d636-41ca-8fa3-422be12130f0 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:03.294971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:03.299169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:03.299169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] PATH_INFO: `/v3/projects/ab16a60e410f4879b7b34b44217baaa6` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:03.299169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab16a60e410f4879b7b34b44217baaa6)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:03.299169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:03.304484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:03.318508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:03.318849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:03.319142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] PATH_INFO: `/v3/projects/683c6772e367473396006f9f21ddd167` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:03.320556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=683c6772e367473396006f9f21ddd167)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:03.322544 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:03.477324 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683c6772e367473396006f9f21ddd167', 'request_id': 'req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.480111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab16a60e410f4879b7b34b44217baaa6', 'request_id': 'req-1cab679a-7388-4125-a1b8-f9921d558aac'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.483410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683c6772e367473396006f9f21ddd167', 'request_id': 'req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.487297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab16a60e410f4879b7b34b44217baaa6', 'request_id': 'req-1cab679a-7388-4125-a1b8-f9921d558aac'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.496742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab16a60e410f4879b7b34b44217baaa6', 'request_id': 'req-1cab679a-7388-4125-a1b8-f9921d558aac'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.499029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683c6772e367473396006f9f21ddd167', 'request_id': 'req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.626881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] The token cache is being invalidate because project ab16a60e410f4879b7b34b44217baaa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:40:03.632750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.632750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1cab679a-7388-4125-a1b8-f9921d558aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.632750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 989/1951] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:40:03 2022] DELETE /identity/v3/projects/ab16a60e410f4879b7b34b44217baaa6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:40:03.637055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514296d0-c65a-4449-b21c-9864d7edb67c None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:03.637772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514296d0-c65a-4449-b21c-9864d7edb67c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:03.638151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514296d0-c65a-4449-b21c-9864d7edb67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:03.660357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-514296d0-c65a-4449-b21c-9864d7edb67c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:03.662187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-514296d0-c65a-4449-b21c-9864d7edb67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:03.666947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:03.734218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 990/1952] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:03.738168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772f03a-c79c-4db8-a794-7f3832dacf9d None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:03.738509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772f03a-c79c-4db8-a794-7f3832dacf9d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:03.745442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1772f03a-c79c-4db8-a794-7f3832dacf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:03.749726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] The token cache is being invalidate because project 683c6772e367473396006f9f21ddd167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:40:03.750034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.751771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:03.754235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 963/1953] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:40:03 2022] DELETE /identity/v3/projects/683c6772e367473396006f9f21ddd167 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:03.762890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:03.765009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:03.781776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:03.792561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-1772f03a-c79c-4db8-a794-7f3832dacf9d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:03.794108 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1772f03a-c79c-4db8-a794-7f3832dacf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:03.805692 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 991/1954] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:03.809731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772f03a-c79c-4db8-a794-7f3832dacf9d None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:03.831757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1772f03a-c79c-4db8-a794-7f3832dacf9d None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:03.841294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:03.842452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:03.842452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:03.842657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:03.843150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:40:03.843456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:03.845997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:03.864929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 992/1955] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:40:03.868268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:03.871453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcebcb-40d7-486b-ad68-3440c5d10e0d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:03.876788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:03.877435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:03.877716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:03.878032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] PATH_INFO: `/v3/users/96af3a0f0e814576a794b6e806118a6a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:03.883456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642d79f-93ca-4a21-8d5b-e9fa6f0e5b85 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:03.887462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] RBAC: Authorizing `identity:delete_user(user_id=96af3a0f0e814576a794b6e806118a6a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:03.890521 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:03.895472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad7cfe-cc31-46cc-9104-1e2e57cd0154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:03.896170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad7cfe-cc31-46cc-9104-1e2e57cd0154 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:03.896452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad7cfe-cc31-46cc-9104-1e2e57cd0154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:03.897062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad7cfe-cc31-46cc-9104-1e2e57cd0154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:03.898952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad7cfe-cc31-46cc-9104-1e2e57cd0154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:03.902644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad7cfe-cc31-46cc-9104-1e2e57cd0154 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:04.045317 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 993/1956] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:40:03 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:04.046471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad7cfe-cc31-46cc-9104-1e2e57cd0154 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:04.149450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96af3a0f0e814576a794b6e806118a6a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:04.189908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96af3a0f0e814576a794b6e806118a6a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:04.202751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad7cfe-cc31-46cc-9104-1e2e57cd0154 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:04.208525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:04.209407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:04.209569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:04.209919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:04.210609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:04.212609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:04.313236 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96af3a0f0e814576a794b6e806118a6a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:04.316106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96af3a0f0e814576a794b6e806118a6a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:04.337849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96af3a0f0e814576a794b6e806118a6a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:04.357740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 964/1957] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:40:03 2022] DELETE /identity/v3/users/96af3a0f0e814576a794b6e806118a6a => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:04.359214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:04.367677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:04.383585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 994/1958] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:40:04 2022] POST /identity/v3/users => generated 292 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:04.390755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:04.390755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:04.414845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e90bc2-642b-41b5-9773-fa8d683f490c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:04.588453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247835b3-0694-4274-83a9-9ffe0a4740af admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:04.596681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bedde2-8190-4e68-8110-ee400a0d3b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:04.597552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bedde2-8190-4e68-8110-ee400a0d3b98 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:04.601724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bedde2-8190-4e68-8110-ee400a0d3b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:04.601724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bedde2-8190-4e68-8110-ee400a0d3b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:04.601724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bedde2-8190-4e68-8110-ee400a0d3b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:04.601724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bedde2-8190-4e68-8110-ee400a0d3b98 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:04.607883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:04.608654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:04.608933 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:04.609207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] PATH_INFO: `/v3/users/7df96c7bf04c4d4f91a5e4903159d5d1` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:04.611024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df96c7bf04c4d4f91a5e4903159d5d1)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:04.614443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:04.626722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 995/1959] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:04.636270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bedde2-8190-4e68-8110-ee400a0d3b98 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:04.659809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bedde2-8190-4e68-8110-ee400a0d3b98 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:04.676016 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e92b2-007e-4174-8769-2b8d01aa12fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:04.676675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e92b2-007e-4174-8769-2b8d01aa12fe admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:04.676997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e92b2-007e-4174-8769-2b8d01aa12fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:04.677356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e92b2-007e-4174-8769-2b8d01aa12fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:04.677927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e92b2-007e-4174-8769-2b8d01aa12fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:04.679608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e92b2-007e-4174-8769-2b8d01aa12fe admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:04.721901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 996/1960] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:04.726361 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e92b2-007e-4174-8769-2b8d01aa12fe admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:04.765027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e92b2-007e-4174-8769-2b8d01aa12fe admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:04.775133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b37b4-6fd3-49b4-bba0-d617590db6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:04.776171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b37b4-6fd3-49b4-bba0-d617590db6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:04.776599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b37b4-6fd3-49b4-bba0-d617590db6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:04.777015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b37b4-6fd3-49b4-bba0-d617590db6fe admin admin] PATH_INFO: `/v3/projects/0c9f0d8dfa9544a9a9fec1c5dad43d74/users/07dec04c01264b90aa962053c6d1304a/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:04.797569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b37b4-6fd3-49b4-bba0-d617590db6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9f0d8dfa9544a9a9fec1c5dad43d74, user_id=07dec04c01264b90aa962053c6d1304a, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:04.804608 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b37b4-6fd3-49b4-bba0-d617590db6fe admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:04.870369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df96c7bf04c4d4f91a5e4903159d5d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:04.910014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 997/1961] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:40:04 2022] PUT /identity/v3/projects/0c9f0d8dfa9544a9a9fec1c5dad43d74/users/07dec04c01264b90aa962053c6d1304a/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:04.910239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df96c7bf04c4d4f91a5e4903159d5d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:04.911598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352913-f4b9-4b6c-b8a7-e1ea03b48ba9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:04.911889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352913-f4b9-4b6c-b8a7-e1ea03b48ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:04.912165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352913-f4b9-4b6c-b8a7-e1ea03b48ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:04.955058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-8b352913-f4b9-4b6c-b8a7-e1ea03b48ba9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:04.956971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8b352913-f4b9-4b6c-b8a7-e1ea03b48ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:05.024533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df96c7bf04c4d4f91a5e4903159d5d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:05.025683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df96c7bf04c4d4f91a5e4903159d5d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:05.040417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d16b2ed4-f436-4735-8011-18ea5c1004d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df96c7bf04c4d4f91a5e4903159d5d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:05.047661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 965/1962] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:40:04 2022] DELETE /identity/v3/users/7df96c7bf04c4d4f91a5e4903159d5d1 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:05.052914 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1594b-8254-4ed0-a63f-cdfc196b7e5d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:05.053391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1594b-8254-4ed0-a63f-cdfc196b7e5d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:05.053820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1594b-8254-4ed0-a63f-cdfc196b7e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:05.079811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 998/1963] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:05.086862 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9db1594b-8254-4ed0-a63f-cdfc196b7e5d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:05.088571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9db1594b-8254-4ed0-a63f-cdfc196b7e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:05.111484 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352913-f4b9-4b6c-b8a7-e1ea03b48ba9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:05.111812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8b352913-f4b9-4b6c-b8a7-e1ea03b48ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:05.221803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 966/1964] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:05.233860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1594b-8254-4ed0-a63f-cdfc196b7e5d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:05.241298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352913-f4b9-4b6c-b8a7-e1ea03b48ba9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:05.248621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9db1594b-8254-4ed0-a63f-cdfc196b7e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:05.321215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:05.322052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:05.322337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:05.322610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:05.332951 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:05.334591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:05.345851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 999/1965] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:40:05.356409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:05.357924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:05.374950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f5a40-9bd5-49cf-8e05-4599dda14b0f service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:05.379998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:05.380640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:05.381002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:05.381242 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] PATH_INFO: `/v3/projects/42cc467a8b2a418999855bd196aa5f03` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:05.382440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] RBAC: Authorizing `identity:delete_project(project_id=42cc467a8b2a418999855bd196aa5f03)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:05.383367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1594b-8254-4ed0-a63f-cdfc196b7e5d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:05.384427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:05.388639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ff8c1-ffed-4913-a6cc-a141f84999f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:05.390076 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff8c1-ffed-4913-a6cc-a141f84999f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:05.390288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff8c1-ffed-4913-a6cc-a141f84999f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:05.390760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ff8c1-ffed-4913-a6cc-a141f84999f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:05.404485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ff8c1-ffed-4913-a6cc-a141f84999f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:05.406719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ff8c1-ffed-4913-a6cc-a141f84999f2 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:05.432750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 967/1966] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:05.612169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42cc467a8b2a418999855bd196aa5f03', 'request_id': 'req-37a611ea-8d85-42d3-84c6-884279fedfce'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:05.619860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42cc467a8b2a418999855bd196aa5f03', 'request_id': 'req-37a611ea-8d85-42d3-84c6-884279fedfce'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:05.628426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42cc467a8b2a418999855bd196aa5f03', 'request_id': 'req-37a611ea-8d85-42d3-84c6-884279fedfce'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:05.773850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] The token cache is being invalidate because project 42cc467a8b2a418999855bd196aa5f03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:40:05.775335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:05.777034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-37a611ea-8d85-42d3-84c6-884279fedfce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:05.780508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1000/1967] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:40:05 2022] DELETE /identity/v3/projects/42cc467a8b2a418999855bd196aa5f03 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:05.785376 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5dd37-5890-4247-9f0e-0fb373cd615b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:05.785724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5dd37-5890-4247-9f0e-0fb373cd615b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:05.786029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5dd37-5890-4247-9f0e-0fb373cd615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:05.824256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7eb5dd37-5890-4247-9f0e-0fb373cd615b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:05.827356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7eb5dd37-5890-4247-9f0e-0fb373cd615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:06.059807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 968/1968] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:06.065877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7d0ec-7c6d-409a-9f09-7b81a64f7bb1 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.066522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7d0ec-7c6d-409a-9f09-7b81a64f7bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.067648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7d0ec-7c6d-409a-9f09-7b81a64f7bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.093092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-fda7d0ec-7c6d-409a-9f09-7b81a64f7bb1 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:06.121767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fda7d0ec-7c6d-409a-9f09-7b81a64f7bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:06.121767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1001/1969] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:06.125094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5dd37-5890-4247-9f0e-0fb373cd615b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:06.136631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5dd37-5890-4247-9f0e-0fb373cd615b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:06.142736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:06.143547 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.143782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.144055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.145489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:40:06.145796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:06.147486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:06.162958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 969/1970] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:40:06.167902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7d0ec-7c6d-409a-9f09-7b81a64f7bb1 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:06.178227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7d0ec-7c6d-409a-9f09-7b81a64f7bb1 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:06.183428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5638c-4d15-42b9-82e4-816f1c5366d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:06.184075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5638c-4d15-42b9-82e4-816f1c5366d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.184398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5638c-4d15-42b9-82e4-816f1c5366d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.184611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5638c-4d15-42b9-82e4-816f1c5366d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.185254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5638c-4d15-42b9-82e4-816f1c5366d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:06.198822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5638c-4d15-42b9-82e4-816f1c5366d5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:06.291660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1002/1971] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:40:06 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:06.295776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:06.310899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9071d-95ea-4d31-a7fd-3cf7bf5a9435 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:06.407254 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00894af-6255-431a-8898-7690e82bbea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:06.407984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00894af-6255-431a-8898-7690e82bbea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.408293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00894af-6255-431a-8898-7690e82bbea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.408628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00894af-6255-431a-8898-7690e82bbea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.409628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00894af-6255-431a-8898-7690e82bbea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:06.411869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00894af-6255-431a-8898-7690e82bbea5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:06.540899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:06.553836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5638c-4d15-42b9-82e4-816f1c5366d5 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:06.554031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 970/1972] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:40:06 2022] POST /identity/v3/users => generated 292 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:06.568600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5638c-4d15-42b9-82e4-816f1c5366d5 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:06.575570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a279a8-050d-41e1-bd9b-eba68be2288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:06.576485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a279a8-050d-41e1-bd9b-eba68be2288e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.576974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a279a8-050d-41e1-bd9b-eba68be2288e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.577507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a279a8-050d-41e1-bd9b-eba68be2288e admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.578269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a279a8-050d-41e1-bd9b-eba68be2288e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:06.579976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a279a8-050d-41e1-bd9b-eba68be2288e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:06.602479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1003/1973] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:40:06.604839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00894af-6255-431a-8898-7690e82bbea5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:06.616073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00894af-6255-431a-8898-7690e82bbea5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:06.624527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:06.625137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.625419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.625697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.626793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:06.628385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:06.640536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 971/1974] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:06.644662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a279a8-050d-41e1-bd9b-eba68be2288e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:06.659884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a279a8-050d-41e1-bd9b-eba68be2288e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:06.677757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9c142-2cd0-4544-81d9-c22699f1d00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:06.678471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c142-2cd0-4544-81d9-c22699f1d00c admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.678758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c142-2cd0-4544-81d9-c22699f1d00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.679187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c142-2cd0-4544-81d9-c22699f1d00c admin admin] PATH_INFO: `/v3/projects/c7cf7a200074438094f340260b45c1fd/users/6580055b620f4322bbb4dfbde1a5d989/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.703036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9c142-2cd0-4544-81d9-c22699f1d00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cf7a200074438094f340260b45c1fd, user_id=6580055b620f4322bbb4dfbde1a5d989, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:06.706647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9c142-2cd0-4544-81d9-c22699f1d00c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:06.725341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:06.727184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:06.750601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada764a-004d-4e74-9a14-acb6de12e8f5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:06.763031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:06.763837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.764187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.764530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] PATH_INFO: `/v3/projects/7b2052c4ed6041ce9d5fc06aa845ed6a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.766374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2052c4ed6041ce9d5fc06aa845ed6a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:06.769359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:06.774603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:06.834328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1004/1975] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:40:06 2022] PUT /identity/v3/projects/c7cf7a200074438094f340260b45c1fd/users/6580055b620f4322bbb4dfbde1a5d989/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:06.845826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716752e-32f2-4b53-9d00-3b1ab048caee None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:06.846253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716752e-32f2-4b53-9d00-3b1ab048caee None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:06.846533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716752e-32f2-4b53-9d00-3b1ab048caee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:06.882280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c716752e-32f2-4b53-9d00-3b1ab048caee None None] MFA Rules not processed for user `6580055b620f4322bbb4dfbde1a5d989`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:06.883647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c716752e-32f2-4b53-9d00-3b1ab048caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:06.940666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2052c4ed6041ce9d5fc06aa845ed6a', 'request_id': 'req-ef7a3774-92c8-4dcd-aa92-2913af54e87f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:06.949127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2052c4ed6041ce9d5fc06aa845ed6a', 'request_id': 'req-ef7a3774-92c8-4dcd-aa92-2913af54e87f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:06.957310 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2052c4ed6041ce9d5fc06aa845ed6a', 'request_id': 'req-ef7a3774-92c8-4dcd-aa92-2913af54e87f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:07.013632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1005/1976] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:07.025044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716752e-32f2-4b53-9d00-3b1ab048caee None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:07.028606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c716752e-32f2-4b53-9d00-3b1ab048caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:07.211919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716752e-32f2-4b53-9d00-3b1ab048caee None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:07.231881 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] The token cache is being invalidate because project 7b2052c4ed6041ce9d5fc06aa845ed6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:40:07.232321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:07.234000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:07.251121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 972/1977] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:40:06 2022] DELETE /identity/v3/projects/7b2052c4ed6041ce9d5fc06aa845ed6a => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:07.334286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e02725-f8f6-4b1d-b109-e83aa7721983 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:07.334967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02725-f8f6-4b1d-b109-e83aa7721983 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:07.335347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02725-f8f6-4b1d-b109-e83aa7721983 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:07.335707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02725-f8f6-4b1d-b109-e83aa7721983 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:07.343180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-71e02725-f8f6-4b1d-b109-e83aa7721983 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:07.386424 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:07.387965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:07.506868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e02725-f8f6-4b1d-b109-e83aa7721983 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:07.509127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e02725-f8f6-4b1d-b109-e83aa7721983 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:07.543102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1006/1978] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:40:07.566224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a3774-92c8-4dcd-aa92-2913af54e87f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:07.571364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:07.572177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:07.572566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:07.572950 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] PATH_INFO: `/v3/users/25f0393cb6cf4e93a4a8fb672ac03696` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:07.580591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f0393cb6cf4e93a4a8fb672ac03696)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:07.583823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:07.820208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f0393cb6cf4e93a4a8fb672ac03696'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:07.838395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f0393cb6cf4e93a4a8fb672ac03696'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:07.947848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f0393cb6cf4e93a4a8fb672ac03696'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:07.948714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f0393cb6cf4e93a4a8fb672ac03696'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:07.960550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f0393cb6cf4e93a4a8fb672ac03696'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:07.972979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 973/1979] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:40:07 2022] DELETE /identity/v3/users/25f0393cb6cf4e93a4a8fb672ac03696 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:07.975690 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7fd51-91f5-4a36-986d-49a4bb7babe8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:07.976117 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7fd51-91f5-4a36-986d-49a4bb7babe8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:07.976486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7fd51-91f5-4a36-986d-49a4bb7babe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:07.998873 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-59f7fd51-91f5-4a36-986d-49a4bb7babe8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:08.000265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-59f7fd51-91f5-4a36-986d-49a4bb7babe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:08.039138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:08.101807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1007/1980] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:08.133831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:08.135372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:08.225793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c5233-7694-488a-aeb8-d1bb8852690a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:08.237312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:08.237312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:08.237312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:08.237312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:08.256438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:08.261410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:08.271458 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 974/1981] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:08.296285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c5558-281f-44fd-8772-ace3544cd7e4 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:08.296570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c5558-281f-44fd-8772-ace3544cd7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:08.296844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c5558-281f-44fd-8772-ace3544cd7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:08.371483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-be3c5558-281f-44fd-8772-ace3544cd7e4 None None] MFA Rules not processed for user `07dec04c01264b90aa962053c6d1304a`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:08.381624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-be3c5558-281f-44fd-8772-ace3544cd7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:08.543718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1008/1982] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:08.555382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:08.557560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:08.729529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b80be-2a1f-47c2-b602-65edefcbc457 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:08.752923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f80dd-5bbc-4dfb-999b-9702a5f755bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:08.754993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f80dd-5bbc-4dfb-999b-9702a5f755bf service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:08.760086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f80dd-5bbc-4dfb-999b-9702a5f755bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:08.760588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f80dd-5bbc-4dfb-999b-9702a5f755bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:08.780919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f80dd-5bbc-4dfb-999b-9702a5f755bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:08.783491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f80dd-5bbc-4dfb-999b-9702a5f755bf service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:08.798203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 975/1983] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:09.164981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c5558-281f-44fd-8772-ace3544cd7e4 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:09.170615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:09.172293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-be3c5558-281f-44fd-8772-ace3544cd7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:09.197190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c5558-281f-44fd-8772-ace3544cd7e4 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:09.203764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:09.204400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:09.204847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:09.205309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] PATH_INFO: `/v3/projects/f431e712685e486eb5731f4b0b4b25a0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:09.206886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f431e712685e486eb5731f4b0b4b25a0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:09.209565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:09.407522 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6653-dec2-4695-a6eb-a909a89a6eb6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:09.407807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6653-dec2-4695-a6eb-a909a89a6eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:09.408833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc6653-dec2-4695-a6eb-a909a89a6eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:09.427837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f431e712685e486eb5731f4b0b4b25a0', 'request_id': 'req-261f2b0b-8d78-4e40-9812-5f2b7146231f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:09.437845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f431e712685e486eb5731f4b0b4b25a0', 'request_id': 'req-261f2b0b-8d78-4e40-9812-5f2b7146231f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:09.446643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-22fc6653-dec2-4695-a6eb-a909a89a6eb6 None None] MFA Rules not processed for user `39543aedd1834f68a9d0a4ef174822c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:09.447356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f431e712685e486eb5731f4b0b4b25a0', 'request_id': 'req-261f2b0b-8d78-4e40-9812-5f2b7146231f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:09.447905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22fc6653-dec2-4695-a6eb-a909a89a6eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:09.517072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 976/1984] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:09.566632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] The token cache is being invalidate because project f431e712685e486eb5731f4b0b4b25a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:40:09.567744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:09.568246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-261f2b0b-8d78-4e40-9812-5f2b7146231f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:09.575090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d21ee4-4399-4bd8-b8ff-1b6b2dae4090 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:09.575456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d21ee4-4399-4bd8-b8ff-1b6b2dae4090 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:09.575801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d21ee4-4399-4bd8-b8ff-1b6b2dae4090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:09.576411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1009/1985] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:40:09 2022] DELETE /identity/v3/projects/f431e712685e486eb5731f4b0b4b25a0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:09.598149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-03d21ee4-4399-4bd8-b8ff-1b6b2dae4090 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:09.599474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-03d21ee4-4399-4bd8-b8ff-1b6b2dae4090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:09.685152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:09.899541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 977/1986] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:09.901675 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd820-d470-4653-84df-1b9dae928346 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:09.902089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd820-d470-4653-84df-1b9dae928346 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:09.902376 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cd820-d470-4653-84df-1b9dae928346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:09.939663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-061cd820-d470-4653-84df-1b9dae928346 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:09.942432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-061cd820-d470-4653-84df-1b9dae928346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:09.956402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1010/1987] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:09.957337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d21ee4-4399-4bd8-b8ff-1b6b2dae4090 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:09.971979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d21ee4-4399-4bd8-b8ff-1b6b2dae4090 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:09.983131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:09.983131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:09.985905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:09.987587 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:09.988377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:40:09.990103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:09.993514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:10.024634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 978/1988] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:40:10.026183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cd820-d470-4653-84df-1b9dae928346 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:10.045983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cd820-d470-4653-84df-1b9dae928346 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:10.059146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afe550-ad56-4bcc-8173-e5cc0c4363c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:10.059778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afe550-ad56-4bcc-8173-e5cc0c4363c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.060305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afe550-ad56-4bcc-8173-e5cc0c4363c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.060597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afe550-ad56-4bcc-8173-e5cc0c4363c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:10.061212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afe550-ad56-4bcc-8173-e5cc0c4363c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:10.063181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afe550-ad56-4bcc-8173-e5cc0c4363c1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:10.241881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1011/1989] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:40:10 2022] POST /identity/v3/projects => generated 371 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:10.246916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:10.261113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78606e-3b31-4c7c-9f74-f7ed981085ea admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:10.364311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1830b-befc-46dd-a488-f1542c1b6a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:10.364937 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1830b-befc-46dd-a488-f1542c1b6a95 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.365215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1830b-befc-46dd-a488-f1542c1b6a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.365482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1830b-befc-46dd-a488-f1542c1b6a95 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:10.366284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1830b-befc-46dd-a488-f1542c1b6a95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:10.368823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1830b-befc-46dd-a488-f1542c1b6a95 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:10.422596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aebb61-7b28-408c-a3d8-a139ee8114d4 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.422985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aebb61-7b28-408c-a3d8-a139ee8114d4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.423444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aebb61-7b28-408c-a3d8-a139ee8114d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:10.457410 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-20aebb61-7b28-408c-a3d8-a139ee8114d4 None None] MFA Rules not processed for user `be7509d5cdca46be91f7259e320efd5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:10.459601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-20aebb61-7b28-408c-a3d8-a139ee8114d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:10.481879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 979/1990] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:40:10 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:40:10.486026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1830b-befc-46dd-a488-f1542c1b6a95 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:10.500261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1830b-befc-46dd-a488-f1542c1b6a95 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:10.505804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9628d-414c-418c-8c38-3ab2c13263bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:10.506408 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9628d-414c-418c-8c38-3ab2c13263bc admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.510751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9628d-414c-418c-8c38-3ab2c13263bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.510751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9628d-414c-418c-8c38-3ab2c13263bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:10.510751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9628d-414c-418c-8c38-3ab2c13263bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:10.510751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d9628d-414c-418c-8c38-3ab2c13263bc admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:10.560192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 980/1991] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:10.564902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9628d-414c-418c-8c38-3ab2c13263bc admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:10.576853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9628d-414c-418c-8c38-3ab2c13263bc admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:10.580948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1012/1992] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:10.584265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3a955-f8b1-41a6-95e4-719961f0b180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:10.585119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3a955-f8b1-41a6-95e4-719961f0b180 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.585688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3a955-f8b1-41a6-95e4-719961f0b180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.586205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3a955-f8b1-41a6-95e4-719961f0b180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:10.587075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3a955-f8b1-41a6-95e4-719961f0b180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:10.589148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3a955-f8b1-41a6-95e4-719961f0b180 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:10.589896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aebb61-7b28-408c-a3d8-a139ee8114d4 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:10.592718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-20aebb61-7b28-408c-a3d8-a139ee8114d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:10.624523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 981/1993] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:40:10.629263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3a955-f8b1-41a6-95e4-719961f0b180 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:10.668453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3a955-f8b1-41a6-95e4-719961f0b180 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:10.683199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c783f-daa3-4509-801f-7617ce10cf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:10.687684 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c783f-daa3-4509-801f-7617ce10cf9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.688416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c783f-daa3-4509-801f-7617ce10cf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.688714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c783f-daa3-4509-801f-7617ce10cf9e admin admin] PATH_INFO: `/v3/projects/ce96ec1a68c249af81c3737fdb2c5531/users/760232d05172490b94e1138607acdd72/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:10.704382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aebb61-7b28-408c-a3d8-a139ee8114d4 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:10.721275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6253a6-263b-46a6-964c-54a6f6d6953f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:10.721910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6253a6-263b-46a6-964c-54a6f6d6953f service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.722211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6253a6-263b-46a6-964c-54a6f6d6953f service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.722482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6253a6-263b-46a6-964c-54a6f6d6953f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:10.731973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c783f-daa3-4509-801f-7617ce10cf9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce96ec1a68c249af81c3737fdb2c5531, user_id=760232d05172490b94e1138607acdd72, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:10.731973 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c783f-daa3-4509-801f-7617ce10cf9e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:10.732183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9e6253a6-263b-46a6-964c-54a6f6d6953f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:10.823491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6253a6-263b-46a6-964c-54a6f6d6953f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:10.826756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6253a6-263b-46a6-964c-54a6f6d6953f service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:10.839453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1013/1994] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:40:10.873166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 982/1995] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:40:10 2022] PUT /identity/v3/projects/ce96ec1a68c249af81c3737fdb2c5531/users/760232d05172490b94e1138607acdd72/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:10.884415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbb424-8ec7-4785-9c94-237bb7bcd2ab None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.884695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbb424-8ec7-4785-9c94-237bb7bcd2ab None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.887054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbb424-8ec7-4785-9c94-237bb7bcd2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:10.929637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a7fbb424-8ec7-4785-9c94-237bb7bcd2ab None None] MFA Rules not processed for user `760232d05172490b94e1138607acdd72`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:10.930930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a7fbb424-8ec7-4785-9c94-237bb7bcd2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:10.975540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3855d08-248b-4f46-8ee7-ea654934df5f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:10.976108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3855d08-248b-4f46-8ee7-ea654934df5f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:10.976239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3855d08-248b-4f46-8ee7-ea654934df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:11.000872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a3855d08-248b-4f46-8ee7-ea654934df5f None None] MFA Rules not processed for user `07dec04c01264b90aa962053c6d1304a`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:11.007601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a3855d08-248b-4f46-8ee7-ea654934df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:11.108516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 983/1996] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:11.108919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1014/1997] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:11.132340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3855d08-248b-4f46-8ee7-ea654934df5f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:11.143307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a3855d08-248b-4f46-8ee7-ea654934df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:11.220949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3855d08-248b-4f46-8ee7-ea654934df5f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:11.225681 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cbcaeb843e470781661f00ec91bffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:11.226553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:11.226836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:11.227109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:11.238733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:11.244382 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:11.254354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 984/1998] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:11.650615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:12.148218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:13.113588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:14.042974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:14.584664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f599bfa-c9a3-4c9d-ba6a-7f3016bfd554 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:14.585127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f599bfa-c9a3-4c9d-ba6a-7f3016bfd554 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:14.585262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f599bfa-c9a3-4c9d-ba6a-7f3016bfd554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:14.628285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7f599bfa-c9a3-4c9d-ba6a-7f3016bfd554 None None] MFA Rules not processed for user `b0a7f349d8ba4bea9a609e7cf881b1b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:14.634162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7f599bfa-c9a3-4c9d-ba6a-7f3016bfd554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:14.743859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1015/1999] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:15.152142 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:16.382091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:19.054099 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:19.055703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:19.127718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771032-231b-4280-909c-860b66b81c22 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:19.128233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771032-231b-4280-909c-860b66b81c22 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:19.128748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771032-231b-4280-909c-860b66b81c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:19.161210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e7771032-231b-4280-909c-860b66b81c22 None None] MFA Rules not processed for user `19d863a45ac14c50b940fbc217b70929`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:19.162453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e7771032-231b-4280-909c-860b66b81c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:19.172432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656d862-f773-4b06-ace0-d9313f10eacc service glance] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:19.180759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:19.180759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:19.180759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:19.180759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] PATH_INFO: `/v3/users/6580055b620f4322bbb4dfbde1a5d989` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:19.181658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6580055b620f4322bbb4dfbde1a5d989)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:19.184633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:19.266287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1016/2000] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:19.388260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:19.402910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6580055b620f4322bbb4dfbde1a5d989'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:19.417114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6580055b620f4322bbb4dfbde1a5d989'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:19.495903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6580055b620f4322bbb4dfbde1a5d989'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:19.496529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6580055b620f4322bbb4dfbde1a5d989'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:19.511079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6580055b620f4322bbb4dfbde1a5d989'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:19.517759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 985/2001] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:40:19 2022] DELETE /identity/v3/users/6580055b620f4322bbb4dfbde1a5d989 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:19.523689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493d431-1976-49c6-9138-3e1f087c4910 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:19.524244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493d431-1976-49c6-9138-3e1f087c4910 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:19.524731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493d431-1976-49c6-9138-3e1f087c4910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:19.547034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7493d431-1976-49c6-9138-3e1f087c4910 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:19.548805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7493d431-1976-49c6-9138-3e1f087c4910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:19.615656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1017/2002] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:19.626140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:19.627885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:19.704859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac330f9a-0ff2-496c-be65-69360e922c32 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:19.708998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa57c2-196b-49f3-9e2c-f2fa16b5293b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:19.709609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa57c2-196b-49f3-9e2c-f2fa16b5293b service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:19.709890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa57c2-196b-49f3-9e2c-f2fa16b5293b service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:19.710250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa57c2-196b-49f3-9e2c-f2fa16b5293b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:19.719455 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa57c2-196b-49f3-9e2c-f2fa16b5293b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:19.720871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa57c2-196b-49f3-9e2c-f2fa16b5293b service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:19.737993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 986/2003] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:20.204724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493d431-1976-49c6-9138-3e1f087c4910 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:20.206262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7493d431-1976-49c6-9138-3e1f087c4910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:20.220644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493d431-1976-49c6-9138-3e1f087c4910 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:20.233451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:20.234159 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:20.234431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:20.234700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] PATH_INFO: `/v3/projects/c7cf7a200074438094f340260b45c1fd` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:20.235899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7cf7a200074438094f340260b45c1fd)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:20.237981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:20.363683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7cf7a200074438094f340260b45c1fd', 'request_id': 'req-b1f4b457-cc5f-482f-a038-a96347efe2a6'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:20.370213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7cf7a200074438094f340260b45c1fd', 'request_id': 'req-b1f4b457-cc5f-482f-a038-a96347efe2a6'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:20.377611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7cf7a200074438094f340260b45c1fd', 'request_id': 'req-b1f4b457-cc5f-482f-a038-a96347efe2a6'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:20.443659 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] The token cache is being invalidate because project c7cf7a200074438094f340260b45c1fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:40:20.443949 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:20.444582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-b1f4b457-cc5f-482f-a038-a96347efe2a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:20.446122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1018/2004] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:40:20 2022] DELETE /identity/v3/projects/c7cf7a200074438094f340260b45c1fd => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:20.451338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa66ad-2c1e-4401-be6e-41b8f9db70c1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:20.451800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa66ad-2c1e-4401-be6e-41b8f9db70c1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:20.451887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa66ad-2c1e-4401-be6e-41b8f9db70c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:20.474338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3efa66ad-2c1e-4401-be6e-41b8f9db70c1 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:20.475590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3efa66ad-2c1e-4401-be6e-41b8f9db70c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:20.539721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 987/2005] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:20.547569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c44b5-a214-4298-8863-32715e2fb635 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:20.549010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c44b5-a214-4298-8863-32715e2fb635 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:20.549010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c44b5-a214-4298-8863-32715e2fb635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:20.564210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:20.571870 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-961c44b5-a214-4298-8863-32715e2fb635 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:20.572178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-961c44b5-a214-4298-8863-32715e2fb635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:20.581293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1019/2006] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:20.591820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa66ad-2c1e-4401-be6e-41b8f9db70c1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:20.603917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa66ad-2c1e-4401-be6e-41b8f9db70c1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:20.609298 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:20.610138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:20.610489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:20.610875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:20.611591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:40:20.611954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:20.615266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:20.634980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 988/2007] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:40:20.644125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c44b5-a214-4298-8863-32715e2fb635 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:20.667722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c44b5-a214-4298-8863-32715e2fb635 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:20.685272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c681aa-8a50-43d9-89a7-51a5c82bae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:20.686272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c681aa-8a50-43d9-89a7-51a5c82bae24 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:20.686537 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c681aa-8a50-43d9-89a7-51a5c82bae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:20.688501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c681aa-8a50-43d9-89a7-51a5c82bae24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:20.688501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c681aa-8a50-43d9-89a7-51a5c82bae24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:20.691251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c681aa-8a50-43d9-89a7-51a5c82bae24 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:20.815004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1020/2008] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:40:20 2022] POST /identity/v3/projects => generated 359 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:20.819918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:20.829909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3021e-29e1-4726-9900-d3a4d020a98b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:20.872859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:20.895740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c6efd-7cb2-419a-8858-f0e3a08cd72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:20.896350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6efd-7cb2-419a-8858-f0e3a08cd72e admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:20.896631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6efd-7cb2-419a-8858-f0e3a08cd72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:20.896906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6efd-7cb2-419a-8858-f0e3a08cd72e admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:20.897571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c6efd-7cb2-419a-8858-f0e3a08cd72e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:20.899518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c6efd-7cb2-419a-8858-f0e3a08cd72e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:21.007096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 989/2009] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:40:20 2022] POST /identity/v3/users => generated 280 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:40:21.011332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c681aa-8a50-43d9-89a7-51a5c82bae24 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:21.022516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c681aa-8a50-43d9-89a7-51a5c82bae24 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:21.027482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7510-2968-4702-a037-8c6aa4dcccad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:21.028212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7510-2968-4702-a037-8c6aa4dcccad admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:21.033625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7510-2968-4702-a037-8c6aa4dcccad admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:21.033625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7510-2968-4702-a037-8c6aa4dcccad admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:21.033625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be7510-2968-4702-a037-8c6aa4dcccad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:21.033625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be7510-2968-4702-a037-8c6aa4dcccad admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:21.042858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1021/2010] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:21.046788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c6efd-7cb2-419a-8858-f0e3a08cd72e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:21.057252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c6efd-7cb2-419a-8858-f0e3a08cd72e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:21.063499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0a82-e29b-4017-b012-88d0ddf22b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:21.064145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0a82-e29b-4017-b012-88d0ddf22b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:21.064691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0a82-e29b-4017-b012-88d0ddf22b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:21.065036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0a82-e29b-4017-b012-88d0ddf22b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:21.065864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0a82-e29b-4017-b012-88d0ddf22b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:21.067577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0a82-e29b-4017-b012-88d0ddf22b51 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:21.079468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 990/2011] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:21.083642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7510-2968-4702-a037-8c6aa4dcccad admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:21.097627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7510-2968-4702-a037-8c6aa4dcccad admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:21.106500 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:21.107211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:21.107487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:21.107755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] PATH_INFO: `/v3/projects/0c84049a0bfc4a668c321296cf02c587/users/cbee2e258f4144c38125bf3ac1f64b51/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:21.125414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c84049a0bfc4a668c321296cf02c587, user_id=cbee2e258f4144c38125bf3ac1f64b51, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:21.128150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:21.205253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1022/2012] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:40:21 2022] PUT /identity/v3/projects/0c84049a0bfc4a668c321296cf02c587/users/cbee2e258f4144c38125bf3ac1f64b51/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:21.216248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a99e6b-c9cb-4027-8b62-6837551d03c8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:21.216629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a99e6b-c9cb-4027-8b62-6837551d03c8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:21.216975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a99e6b-c9cb-4027-8b62-6837551d03c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:21.245366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f6a99e6b-c9cb-4027-8b62-6837551d03c8 None None] MFA Rules not processed for user `cbee2e258f4144c38125bf3ac1f64b51`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:21.246705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f6a99e6b-c9cb-4027-8b62-6837551d03c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:21.358199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 991/2013] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:21.409216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:21.410920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:21.468193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7401db3-103c-4fc8-8ec6-9f087827f976 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:21.471787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491803-4576-4884-b29e-27c8bb98328c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:21.472390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491803-4576-4884-b29e-27c8bb98328c service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:21.472664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491803-4576-4884-b29e-27c8bb98328c service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:21.472932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491803-4576-4884-b29e-27c8bb98328c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:21.481567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491803-4576-4884-b29e-27c8bb98328c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:21.483070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491803-4576-4884-b29e-27c8bb98328c service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:21.491780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1023/2014] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:21.508972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a99e6b-c9cb-4027-8b62-6837551d03c8 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:21.576798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a99e6b-c9cb-4027-8b62-6837551d03c8 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:21.581816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:21.582484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:21.582818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:21.583086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] PATH_INFO: `/v3/users/cbee2e258f4144c38125bf3ac1f64b51` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:21.584838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbee2e258f4144c38125bf3ac1f64b51)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:21.586841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:21.796962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbee2e258f4144c38125bf3ac1f64b51'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:21.798415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:21.821899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbee2e258f4144c38125bf3ac1f64b51'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:21.907410 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbee2e258f4144c38125bf3ac1f64b51'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:21.908075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbee2e258f4144c38125bf3ac1f64b51'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:21.916739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbee2e258f4144c38125bf3ac1f64b51'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:21.923771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 992/2015] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:40:21 2022] DELETE /identity/v3/users/cbee2e258f4144c38125bf3ac1f64b51 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:21.929055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be7b2-1041-474f-a055-6d0273ecede2 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:21.929339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be7b2-1041-474f-a055-6d0273ecede2 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:21.929647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be7b2-1041-474f-a055-6d0273ecede2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:21.948593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9c1be7b2-1041-474f-a055-6d0273ecede2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:21.949804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9c1be7b2-1041-474f-a055-6d0273ecede2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:22.015171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1024/2016] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:22.028339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:22.030378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:22.110764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11913003-c6ec-446e-8ce5-4dd4cc2ed27d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:22.115948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67521c44-5262-40f9-a9bd-10bd9b33799d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:22.117350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67521c44-5262-40f9-a9bd-10bd9b33799d service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:22.117731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67521c44-5262-40f9-a9bd-10bd9b33799d service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:22.118487 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67521c44-5262-40f9-a9bd-10bd9b33799d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:22.130533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67521c44-5262-40f9-a9bd-10bd9b33799d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:22.132053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67521c44-5262-40f9-a9bd-10bd9b33799d service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:22.141902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 993/2017] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:22.771169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1be7b2-1041-474f-a055-6d0273ecede2 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:22.772244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9c1be7b2-1041-474f-a055-6d0273ecede2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:22.786898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1be7b2-1041-474f-a055-6d0273ecede2 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:22.792868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:22.793502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:22.793921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:22.794224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] PATH_INFO: `/v3/projects/0c84049a0bfc4a668c321296cf02c587` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:22.795510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c84049a0bfc4a668c321296cf02c587)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:22.798597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:22.921251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:22.927786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c84049a0bfc4a668c321296cf02c587', 'request_id': 'req-e7e1d583-465a-4ea6-ac65-c3110324107f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:22.933955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c84049a0bfc4a668c321296cf02c587', 'request_id': 'req-e7e1d583-465a-4ea6-ac65-c3110324107f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:22.940948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c84049a0bfc4a668c321296cf02c587', 'request_id': 'req-e7e1d583-465a-4ea6-ac65-c3110324107f'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:22.998880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] The token cache is being invalidate because project 0c84049a0bfc4a668c321296cf02c587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:40:22.999152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:22.999797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e7e1d583-465a-4ea6-ac65-c3110324107f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:23.001303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1025/2018] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:40:22 2022] DELETE /identity/v3/projects/0c84049a0bfc4a668c321296cf02c587 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:23.006385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6cca-8a4c-42be-a60f-5a140201a955 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.006847 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6cca-8a4c-42be-a60f-5a140201a955 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.006941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d6cca-8a4c-42be-a60f-5a140201a955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.027253 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-189d6cca-8a4c-42be-a60f-5a140201a955 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:23.031290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-189d6cca-8a4c-42be-a60f-5a140201a955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:23.092907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 994/2019] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:23.102656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a917-fb8e-4d72-9672-98b7ecf55677 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.102942 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a917-fb8e-4d72-9672-98b7ecf55677 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.103217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a917-fb8e-4d72-9672-98b7ecf55677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.125178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-dad0a917-fb8e-4d72-9672-98b7ecf55677 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:23.126581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-dad0a917-fb8e-4d72-9672-98b7ecf55677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:23.135371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1026/2020] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:23.141329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d6cca-8a4c-42be-a60f-5a140201a955 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:23.159497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d6cca-8a4c-42be-a60f-5a140201a955 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:23.165171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:23.165787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.166095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.166368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.167020 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:40:23.167338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:23.169102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:23.185270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 995/2021] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:40:23.185899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0a917-fb8e-4d72-9672-98b7ecf55677 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:23.195499 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0a917-fb8e-4d72-9672-98b7ecf55677 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:23.200395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce1275-8a86-4411-865e-05985f33e7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:23.200904 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce1275-8a86-4411-865e-05985f33e7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.201182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce1275-8a86-4411-865e-05985f33e7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.201451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce1275-8a86-4411-865e-05985f33e7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.202157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce1275-8a86-4411-865e-05985f33e7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:23.204283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce1275-8a86-4411-865e-05985f33e7d0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:23.292425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1027/2022] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:40:23 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:40:23.296693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:23.307456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61c01d-1fdf-46b4-9f97-276eb44a42c8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:23.360404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:23.367244 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36693511-0489-4b29-9c6b-4be15770e2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:23.368197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36693511-0489-4b29-9c6b-4be15770e2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.368421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36693511-0489-4b29-9c6b-4be15770e2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.368689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36693511-0489-4b29-9c6b-4be15770e2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.369373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36693511-0489-4b29-9c6b-4be15770e2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:23.371392 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36693511-0489-4b29-9c6b-4be15770e2c7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:23.450457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 996/2023] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:40:23 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:40:23.454754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce1275-8a86-4411-865e-05985f33e7d0 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:23.466651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce1275-8a86-4411-865e-05985f33e7d0 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:23.472529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf73103-c46e-4ff5-95f7-0d7ebf4e977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:23.473140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf73103-c46e-4ff5-95f7-0d7ebf4e977e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.473423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf73103-c46e-4ff5-95f7-0d7ebf4e977e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.475005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf73103-c46e-4ff5-95f7-0d7ebf4e977e admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.475005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf73103-c46e-4ff5-95f7-0d7ebf4e977e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:23.476046 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf73103-c46e-4ff5-95f7-0d7ebf4e977e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:23.486560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1028/2024] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:23.491538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36693511-0489-4b29-9c6b-4be15770e2c7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:23.502065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36693511-0489-4b29-9c6b-4be15770e2c7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:23.507745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07601ae6-fa40-455c-9eb7-9ee3c3d4e185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:23.508379 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07601ae6-fa40-455c-9eb7-9ee3c3d4e185 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.508657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07601ae6-fa40-455c-9eb7-9ee3c3d4e185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.508923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07601ae6-fa40-455c-9eb7-9ee3c3d4e185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.509565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07601ae6-fa40-455c-9eb7-9ee3c3d4e185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:23.511263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07601ae6-fa40-455c-9eb7-9ee3c3d4e185 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:23.522953 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 997/2025] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:40:23.527025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf73103-c46e-4ff5-95f7-0d7ebf4e977e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:23.537364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf73103-c46e-4ff5-95f7-0d7ebf4e977e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:23.542254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:23.542902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.543181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.543646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] PATH_INFO: `/v3/projects/827d3a5990a940049bcb35a219efdef3/users/c96f2271cf394de1a15fcdfe3d62860a/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.562166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=827d3a5990a940049bcb35a219efdef3, user_id=c96f2271cf394de1a15fcdfe3d62860a, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:23.564816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:23.656100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1029/2026] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:40:23 2022] PUT /identity/v3/projects/827d3a5990a940049bcb35a219efdef3/users/c96f2271cf394de1a15fcdfe3d62860a/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:23.661013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83a71d-3508-4295-889e-b0765d923056 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.661295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83a71d-3508-4295-889e-b0765d923056 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.661566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83a71d-3508-4295-889e-b0765d923056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.681992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bd83a71d-3508-4295-889e-b0765d923056 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:23.683425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bd83a71d-3508-4295-889e-b0765d923056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:23.747406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 998/2027] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:23.760428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:23.763031 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:23.835022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d235f-9b37-4f89-b964-606864408904 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:23.839197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:23.840146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:23.840547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:23.841030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:23.870383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:23.871854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:23.882198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1030/2028] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:25.349394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77074-b595-49d9-bf2d-5e0a5548879b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:25.350638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77074-b595-49d9-bf2d-5e0a5548879b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:25.350735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77074-b595-49d9-bf2d-5e0a5548879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:25.393945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e7c77074-b595-49d9-bf2d-5e0a5548879b None None] MFA Rules not processed for user `c96f2271cf394de1a15fcdfe3d62860a`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:25.395475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e7c77074-b595-49d9-bf2d-5e0a5548879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:25.496744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 999/2029] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:25.510549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:25.512053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:25.573853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8b768-9750-4210-91bc-d1bd1373712a service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:25.577427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b47c-db39-4c72-b46d-49354867124c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:25.578173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b47c-db39-4c72-b46d-49354867124c service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:25.578448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b47c-db39-4c72-b46d-49354867124c service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:25.578721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b47c-db39-4c72-b46d-49354867124c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:25.587735 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b47c-db39-4c72-b46d-49354867124c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:25.589148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b47c-db39-4c72-b46d-49354867124c service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:25.598536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1031/2030] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:27.385237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c355-0719-46b8-9076-1a0e3c20c02d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:27.385701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c355-0719-46b8-9076-1a0e3c20c02d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:27.385792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c355-0719-46b8-9076-1a0e3c20c02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:27.405049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ac92c355-0719-46b8-9076-1a0e3c20c02d None None] MFA Rules not processed for user `c96f2271cf394de1a15fcdfe3d62860a`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:27.406260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ac92c355-0719-46b8-9076-1a0e3c20c02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:27.413096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1000/2031] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:31.213465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f31826-a415-44bb-b786-e210c43f6489 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:31.214004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f31826-a415-44bb-b786-e210c43f6489 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:31.214103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f31826-a415-44bb-b786-e210c43f6489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:31.234099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-95f31826-a415-44bb-b786-e210c43f6489 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:31.235509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-95f31826-a415-44bb-b786-e210c43f6489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:31.244593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:31.296626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1032/2032] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:40:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:31.347730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92c355-0719-46b8-9076-1a0e3c20c02d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:31.349109 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ac92c355-0719-46b8-9076-1a0e3c20c02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:31.417641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:31.434077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92c355-0719-46b8-9076-1a0e3c20c02d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:31.450897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488755e-dbc8-4ec7-a738-4c5521da75c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:31.451585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488755e-dbc8-4ec7-a738-4c5521da75c0 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:31.451868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488755e-dbc8-4ec7-a738-4c5521da75c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:31.452140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488755e-dbc8-4ec7-a738-4c5521da75c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:31.461475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488755e-dbc8-4ec7-a738-4c5521da75c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:31.463102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488755e-dbc8-4ec7-a738-4c5521da75c0 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:31.471416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1001/2033] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:40:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:40:41.495950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=20 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:41.634714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=6 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:53.675888 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f31826-a415-44bb-b786-e210c43f6489 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:53.677503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-95f31826-a415-44bb-b786-e210c43f6489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:53.691985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f31826-a415-44bb-b786-e210c43f6489 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:53.696636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:53.698058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:53.698343 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:53.698617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] PATH_INFO: `/v3/users/760232d05172490b94e1138607acdd72` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:53.699772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=760232d05172490b94e1138607acdd72)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:53.701712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:53.916991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760232d05172490b94e1138607acdd72'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:53.930133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760232d05172490b94e1138607acdd72'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:53.930769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760232d05172490b94e1138607acdd72'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:54.075599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760232d05172490b94e1138607acdd72'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:54.087706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760232d05172490b94e1138607acdd72'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:54.090825 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1033/2034] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:40:53 2022] DELETE /identity/v3/users/760232d05172490b94e1138607acdd72 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:40:54.095502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb926bb-702f-426a-aa8e-a993d9eccfdf None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:54.096018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb926bb-702f-426a-aa8e-a993d9eccfdf None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:54.096018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb926bb-702f-426a-aa8e-a993d9eccfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:54.116384 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1cb926bb-702f-426a-aa8e-a993d9eccfdf None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:54.117594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1cb926bb-702f-426a-aa8e-a993d9eccfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:54.193657 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1002/2035] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:54.203181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:54.204708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:54.273860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b2ae4-91d9-445f-87dd-ecb2ca4a6535 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:54.277563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901fc69-5b47-48dc-8d19-33cc51afa91b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:54.280542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901fc69-5b47-48dc-8d19-33cc51afa91b service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:54.280542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901fc69-5b47-48dc-8d19-33cc51afa91b service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:54.280542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901fc69-5b47-48dc-8d19-33cc51afa91b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:54.288946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901fc69-5b47-48dc-8d19-33cc51afa91b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:54.295332 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901fc69-5b47-48dc-8d19-33cc51afa91b service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:54.308487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1034/2036] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:54.917703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb926bb-702f-426a-aa8e-a993d9eccfdf None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:54.919636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1cb926bb-702f-426a-aa8e-a993d9eccfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:54.933372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb926bb-702f-426a-aa8e-a993d9eccfdf None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:54.938192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:54.938855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:54.939130 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:54.939400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] PATH_INFO: `/v3/projects/ce96ec1a68c249af81c3737fdb2c5531` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:54.940545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce96ec1a68c249af81c3737fdb2c5531)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:54.942634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:55.132036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce96ec1a68c249af81c3737fdb2c5531', 'request_id': 'req-794d4089-eae6-4f45-99d6-365bb44e267e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:55.138866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce96ec1a68c249af81c3737fdb2c5531', 'request_id': 'req-794d4089-eae6-4f45-99d6-365bb44e267e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:55.145442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce96ec1a68c249af81c3737fdb2c5531', 'request_id': 'req-794d4089-eae6-4f45-99d6-365bb44e267e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:55.226474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] The token cache is being invalidate because project ce96ec1a68c249af81c3737fdb2c5531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:40:55.226729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:55.227354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-794d4089-eae6-4f45-99d6-365bb44e267e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:40:55.228865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1003/2037] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:40:54 2022] DELETE /identity/v3/projects/ce96ec1a68c249af81c3737fdb2c5531 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:40:57.312412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1da4e6-593b-4454-8a00-f884a72dcaa7 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:57.312849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1da4e6-593b-4454-8a00-f884a72dcaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:57.313058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1da4e6-593b-4454-8a00-f884a72dcaa7 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:57.314656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1035/2038] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:40:57.320573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6730e-c957-47d1-818c-dfd681e9bbf8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:57.321613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6730e-c957-47d1-818c-dfd681e9bbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:57.325580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6730e-c957-47d1-818c-dfd681e9bbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:57.351299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-22e6730e-c957-47d1-818c-dfd681e9bbf8 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:57.352538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22e6730e-c957-47d1-818c-dfd681e9bbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:57.430086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1004/2039] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:57.445066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1da4e6-593b-4454-8a00-f884a72dcaa7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:57.445066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bb1da4e6-593b-4454-8a00-f884a72dcaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:57.513630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1da4e6-593b-4454-8a00-f884a72dcaa7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:57.517295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2c0bf-624c-4c0e-acfb-b03029684d20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cbcaeb843e470781661f00ec91bffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:57.517993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2c0bf-624c-4c0e-acfb-b03029684d20 service glance] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:57.518348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2c0bf-624c-4c0e-acfb-b03029684d20 service glance] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:57.518627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2c0bf-624c-4c0e-acfb-b03029684d20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:57.529320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2c0bf-624c-4c0e-acfb-b03029684d20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:57.530805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2c0bf-624c-4c0e-acfb-b03029684d20 service glance] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:57.542274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1036/2040] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:57.751380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e852b-5112-4846-9633-597e2b2c6976 None None] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:57.751766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e852b-5112-4846-9633-597e2b2c6976 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:57.752043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e852b-5112-4846-9633-597e2b2c6976 None None] PATH_INFO: `/` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:57.753614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1005/2041] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:40:57.758444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d998d-0483-4945-9527-44255f493bbc None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:57.758803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d998d-0483-4945-9527-44255f493bbc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:57.759146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d998d-0483-4945-9527-44255f493bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:57.783430 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-4e1d998d-0483-4945-9527-44255f493bbc None None] MFA Rules not processed for user `db04575738b04f2d95803498e19554c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:57.784657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4e1d998d-0483-4945-9527-44255f493bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:57.798973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:57.841916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1037/2042] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:57.854062 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e852b-5112-4846-9633-597e2b2c6976 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:57.855939 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-af0e852b-5112-4846-9633-597e2b2c6976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:57.869864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e852b-5112-4846-9633-597e2b2c6976 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:57.873731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a49243-98b7-4c97-999d-f7484175f132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:57.874459 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a49243-98b7-4c97-999d-f7484175f132 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:57.874760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a49243-98b7-4c97-999d-f7484175f132 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:57.875031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a49243-98b7-4c97-999d-f7484175f132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:57.884580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a49243-98b7-4c97-999d-f7484175f132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:57.886621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a49243-98b7-4c97-999d-f7484175f132 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:57.895949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1006/2043] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:40:57.987249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171da5-b233-4d63-9b6d-c3182c1ddfbb None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:57.993134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171da5-b233-4d63-9b6d-c3182c1ddfbb None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:57.993134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171da5-b233-4d63-9b6d-c3182c1ddfbb None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:57.993134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1038/2044] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:40:57.996200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd812c8f-dfc1-45fb-855c-09bdfc7ce5a8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:57.996488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd812c8f-dfc1-45fb-855c-09bdfc7ce5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:57.996783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd812c8f-dfc1-45fb-855c-09bdfc7ce5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:58.004841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:58.018274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bd812c8f-dfc1-45fb-855c-09bdfc7ce5a8 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:58.020611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bd812c8f-dfc1-45fb-855c-09bdfc7ce5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:58.029707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1007/2045] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:40:58.041156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c171da5-b233-4d63-9b6d-c3182c1ddfbb None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:40:58.054363 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c171da5-b233-4d63-9b6d-c3182c1ddfbb None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:40:58.058087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:40:58.058686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:58.058968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:58.059241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:58.073790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:40:58.073914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:40:58.085604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1039/2046] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:40:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:08.216596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=21 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:08.296393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:12.092334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e868c-8308-4d2d-8d0c-99b3dc08fe48 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:12.092913 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e868c-8308-4d2d-8d0c-99b3dc08fe48 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:12.092913 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e868c-8308-4d2d-8d0c-99b3dc08fe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:12.112445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-db8e868c-8308-4d2d-8d0c-99b3dc08fe48 None None] MFA Rules not processed for user `c96f2271cf394de1a15fcdfe3d62860a`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:12.113626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-db8e868c-8308-4d2d-8d0c-99b3dc08fe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:12.171909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1008/2047] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:41:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:14.788588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:14.790054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:14.859860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d166e9e-3df5-4c03-8941-3d548d1aa014 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:14.864607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:14.865207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:14.865483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:14.865753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:14.867538 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:14.879647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:14.881192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:14.891744 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1040/2048] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:19.905906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bf917-ca23-42dc-83bc-549a0ed5d62f None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:19.906409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bf917-ca23-42dc-83bc-549a0ed5d62f None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:19.906499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bf917-ca23-42dc-83bc-549a0ed5d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:19.927873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b52bf917-ca23-42dc-83bc-549a0ed5d62f None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:19.930182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b52bf917-ca23-42dc-83bc-549a0ed5d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:19.992502 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1009/2049] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:41:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:20.007802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:20.009554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:20.097837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbf4e4-72ca-40ce-b920-bdecd6eb7dc7 service placement] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:20.104549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a53da7-c90c-4a7c-9a4f-0088ecfbb1dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:20.106013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a53da7-c90c-4a7c-9a4f-0088ecfbb1dc service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:20.106252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a53da7-c90c-4a7c-9a4f-0088ecfbb1dc service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:20.106568 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a53da7-c90c-4a7c-9a4f-0088ecfbb1dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:20.117864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a53da7-c90c-4a7c-9a4f-0088ecfbb1dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:20.122170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a53da7-c90c-4a7c-9a4f-0088ecfbb1dc service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:20.129251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1041/2050] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:41:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:41:29.985005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=50 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:30.125752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=64 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:39.670278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bf917-ca23-42dc-83bc-549a0ed5d62f None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:39.679541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bf917-ca23-42dc-83bc-549a0ed5d62f None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:39.683162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c20db-8a7e-4828-b57d-c8a6bd1e1c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:39.683766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c20db-8a7e-4828-b57d-c8a6bd1e1c9a service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:39.684048 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c20db-8a7e-4828-b57d-c8a6bd1e1c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:39.684331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c20db-8a7e-4828-b57d-c8a6bd1e1c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:39.686607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-292c20db-8a7e-4828-b57d-c8a6bd1e1c9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:39.753505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c20db-8a7e-4828-b57d-c8a6bd1e1c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:39.754954 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c20db-8a7e-4828-b57d-c8a6bd1e1c9a service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:39.769275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1010/2051] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:49.764922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=30 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:52.270402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4dc1e-95c9-4000-a0ac-da1e1f7a389b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.270900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4dc1e-95c9-4000-a0ac-da1e1f7a389b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.271010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4dc1e-95c9-4000-a0ac-da1e1f7a389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.292280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-40f4dc1e-95c9-4000-a0ac-da1e1f7a389b None None] MFA Rules not processed for user `07dec04c01264b90aa962053c6d1304a`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:52.293472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-40f4dc1e-95c9-4000-a0ac-da1e1f7a389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:52.348250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1042/2052] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:53.651686 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcfac1-f413-4eb1-8c03-7dc0bf71bcd4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:53.652157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcfac1-f413-4eb1-8c03-7dc0bf71bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:53.652247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcfac1-f413-4eb1-8c03-7dc0bf71bcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:53.671790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-dbdcfac1-f413-4eb1-8c03-7dc0bf71bcd4 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:53.672975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dbdcfac1-f413-4eb1-8c03-7dc0bf71bcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:53.680723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1011/2053] 10.222.0.44 () {58 vars in 967 bytes} [Wed Oct 12 12:41:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:53.719976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4dc1e-95c9-4000-a0ac-da1e1f7a389b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:53.732841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4dc1e-95c9-4000-a0ac-da1e1f7a389b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:53.738111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebc74a-7a4b-4bd6-9d74-09b127e3d6ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:53.738719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc74a-7a4b-4bd6-9d74-09b127e3d6ad service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:53.738994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc74a-7a4b-4bd6-9d74-09b127e3d6ad service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:53.739259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc74a-7a4b-4bd6-9d74-09b127e3d6ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:53.748553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebc74a-7a4b-4bd6-9d74-09b127e3d6ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:53.750171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebc74a-7a4b-4bd6-9d74-09b127e3d6ad service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:53.758426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1043/2054] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:41:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:42:01.238761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca426-89b4-4a37-9808-0f30d9c0abd5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:01.244959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca426-89b4-4a37-9808-0f30d9c0abd5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:01.244959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca426-89b4-4a37-9808-0f30d9c0abd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:01.263081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b26ca426-89b4-4a37-9808-0f30d9c0abd5 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:01.267870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b26ca426-89b4-4a37-9808-0f30d9c0abd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:01.273182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1012/2055] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:42:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:42:01.286880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebc74a-7a4b-4bd6-9d74-09b127e3d6ad service swift] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:01.298556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebc74a-7a4b-4bd6-9d74-09b127e3d6ad service swift] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:01.304133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7d32d-7bad-4614-a6e7-4a174ec25d12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:01.304728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7d32d-7bad-4614-a6e7-4a174ec25d12 service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:01.305004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7d32d-7bad-4614-a6e7-4a174ec25d12 service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:01.305272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7d32d-7bad-4614-a6e7-4a174ec25d12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:01.314403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7d32d-7bad-4614-a6e7-4a174ec25d12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:01.316070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7d32d-7bad-4614-a6e7-4a174ec25d12 service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:01.324830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1044/2056] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:42:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:42:11.257799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=6 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:11.322406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=37 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:14.713877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f81a73-452a-43c9-8899-adba151412bc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:14.714425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f81a73-452a-43c9-8899-adba151412bc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:14.715434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f81a73-452a-43c9-8899-adba151412bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:14.738660 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-96f81a73-452a-43c9-8899-adba151412bc None None] MFA Rules not processed for user `39543aedd1834f68a9d0a4ef174822c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:14.739893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-96f81a73-452a-43c9-8899-adba151412bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:14.805073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1013/2057] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:42:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:19.458266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ad978-3def-4758-91e2-5975751534a8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:19.458817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ad978-3def-4758-91e2-5975751534a8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:19.458817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ad978-3def-4758-91e2-5975751534a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:19.478745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2c4ad978-3def-4758-91e2-5975751534a8 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:19.479881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2c4ad978-3def-4758-91e2-5975751534a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:19.487436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1045/2058] 10.222.0.44 () {58 vars in 967 bytes} [Wed Oct 12 12:42:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:19.500543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f81a73-452a-43c9-8899-adba151412bc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:19.518302 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f81a73-452a-43c9-8899-adba151412bc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:19.524338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2812933-61f4-478c-ad13-d6f39f3f3c0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:19.525473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2812933-61f4-478c-ad13-d6f39f3f3c0b service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:19.525901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2812933-61f4-478c-ad13-d6f39f3f3c0b service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:19.526393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2812933-61f4-478c-ad13-d6f39f3f3c0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:19.537345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2812933-61f4-478c-ad13-d6f39f3f3c0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:19.539011 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2812933-61f4-478c-ad13-d6f39f3f3c0b service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:19.547695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1014/2059] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:42:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 12:42:24.855737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ad978-3def-4758-91e2-5975751534a8 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:24.857174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2c4ad978-3def-4758-91e2-5975751534a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:24.926865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ad978-3def-4758-91e2-5975751534a8 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:24.931530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:24.932084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:24.932370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:24.932649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] PATH_INFO: `/v3/users/c96f2271cf394de1a15fcdfe3d62860a` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:24.933789 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96f2271cf394de1a15fcdfe3d62860a)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:24.935758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:25.141546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96f2271cf394de1a15fcdfe3d62860a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.154423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96f2271cf394de1a15fcdfe3d62860a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.155152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96f2271cf394de1a15fcdfe3d62860a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.212638 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96f2271cf394de1a15fcdfe3d62860a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.220245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5938d7dc-9552-4586-acdb-3887c69d172f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96f2271cf394de1a15fcdfe3d62860a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.226151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1046/2060] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:42:24 2022] DELETE /identity/v3/users/c96f2271cf394de1a15fcdfe3d62860a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:42:25.436902 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2812933-61f4-478c-ad13-d6f39f3f3c0b service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:25.448682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2812933-61f4-478c-ad13-d6f39f3f3c0b service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:25.506216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:25.506843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:25.507119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:25.507390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] PATH_INFO: `/v3/projects/827d3a5990a940049bcb35a219efdef3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:25.508515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=827d3a5990a940049bcb35a219efdef3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:25.510437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:25.617947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '827d3a5990a940049bcb35a219efdef3', 'request_id': 'req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.623744 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '827d3a5990a940049bcb35a219efdef3', 'request_id': 'req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.630091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '827d3a5990a940049bcb35a219efdef3', 'request_id': 'req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.686294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] The token cache is being invalidate because project 827d3a5990a940049bcb35a219efdef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:42:25.686516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.687749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-93c1bd3e-a3ad-4ca3-834e-7c4b23133c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:25.689221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1015/2061] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:42:25 2022] DELETE /identity/v3/projects/827d3a5990a940049bcb35a219efdef3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:42:32.211628 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139824-c91f-4bc8-a2e2-ca46e6efd5ae None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:32.212135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139824-c91f-4bc8-a2e2-ca46e6efd5ae None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:32.212234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139824-c91f-4bc8-a2e2-ca46e6efd5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:32.231660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-39139824-c91f-4bc8-a2e2-ca46e6efd5ae None None] MFA Rules not processed for user `19d863a45ac14c50b940fbc217b70929`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:32.232846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-39139824-c91f-4bc8-a2e2-ca46e6efd5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:32.286709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1047/2062] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:42:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:35.683271 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:42.279238 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:00.312183 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1b650-ef82-4608-93cf-b9d76298d073 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:00.312634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1b650-ef82-4608-93cf-b9d76298d073 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:00.312746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1b650-ef82-4608-93cf-b9d76298d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:00.334102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-25b1b650-ef82-4608-93cf-b9d76298d073 None None] MFA Rules not processed for user `df8da1fe608d4dd6acb6b7298ca56d45`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:00.334412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-25b1b650-ef82-4608-93cf-b9d76298d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:00.389989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1016/2063] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:02.167475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39139824-c91f-4bc8-a2e2-ca46e6efd5ae None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:02.168987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-39139824-c91f-4bc8-a2e2-ca46e6efd5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:02.230279 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39139824-c91f-4bc8-a2e2-ca46e6efd5ae None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:02.233827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:02.234446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:02.235218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:02.235218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:02.236779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:02.310625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:02.312073 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:02.324411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1048/2064] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:03.441051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1b650-ef82-4608-93cf-b9d76298d073 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:03.442542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-25b1b650-ef82-4608-93cf-b9d76298d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:03.455802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1b650-ef82-4608-93cf-b9d76298d073 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:03.459779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:03.462724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:03.462724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:03.462724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:03.463593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:03.478513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:03.479727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:03.491661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1017/2065] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:08.531561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:08.532040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:08.601754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ddd8f-aa5f-46e6-8176-abbe909ee941 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:08.606566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:08.607184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:08.607462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:08.607736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] PATH_INFO: `/v3/users/07dec04c01264b90aa962053c6d1304a` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:08.608870 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07dec04c01264b90aa962053c6d1304a)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:08.610814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:08.808858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:08.811400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07dec04c01264b90aa962053c6d1304a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:08.822992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07dec04c01264b90aa962053c6d1304a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:08.823643 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07dec04c01264b90aa962053c6d1304a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:08.922358 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07dec04c01264b90aa962053c6d1304a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:08.930421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1988f25d-4cbc-4f26-a0e8-1c7fe99142d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07dec04c01264b90aa962053c6d1304a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:08.936283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1049/2066] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:08 2022] DELETE /identity/v3/users/07dec04c01264b90aa962053c6d1304a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:09.158134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:09.169538 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008191f2-c148-48d5-9f3d-3d17a9996bfe service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:09.228118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:09.228747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.229051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.229321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] PATH_INFO: `/v3/projects/0c9f0d8dfa9544a9a9fec1c5dad43d74` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.230462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9f0d8dfa9544a9a9fec1c5dad43d74)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:09.232344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:09.324039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9f0d8dfa9544a9a9fec1c5dad43d74', 'request_id': 'req-260d96a7-515f-4794-b738-f1c9bc871251'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:09.329900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9f0d8dfa9544a9a9fec1c5dad43d74', 'request_id': 'req-260d96a7-515f-4794-b738-f1c9bc871251'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:09.336821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9f0d8dfa9544a9a9fec1c5dad43d74', 'request_id': 'req-260d96a7-515f-4794-b738-f1c9bc871251'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:09.408110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] The token cache is being invalidate because project 0c9f0d8dfa9544a9a9fec1c5dad43d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:09.414755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:09.414755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-260d96a7-515f-4794-b738-f1c9bc871251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:09.414755 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1018/2067] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:09 2022] DELETE /identity/v3/projects/0c9f0d8dfa9544a9a9fec1c5dad43d74 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:09.416269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bf25f-da87-4edf-91ad-4a2cf9c981ca None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.416560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bf25f-da87-4edf-91ad-4a2cf9c981ca None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.416895 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bf25f-da87-4edf-91ad-4a2cf9c981ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.446370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-973bf25f-da87-4edf-91ad-4a2cf9c981ca None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:09.446370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-973bf25f-da87-4edf-91ad-4a2cf9c981ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:09.505473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1050/2068] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:09.513991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944469-e8e7-4d90-8ee0-00d054642a94 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.514283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944469-e8e7-4d90-8ee0-00d054642a94 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.514560 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7944469-e8e7-4d90-8ee0-00d054642a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.535339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-c7944469-e8e7-4d90-8ee0-00d054642a94 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:09.536530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c7944469-e8e7-4d90-8ee0-00d054642a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:09.545268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1019/2069] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:09.549374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bf25f-da87-4edf-91ad-4a2cf9c981ca None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:09.559651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bf25f-da87-4edf-91ad-4a2cf9c981ca None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:09.564203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:09.564797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.565094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.565369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.566025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:09.566334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:09.567924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:09.579560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1051/2070] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:09.583599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944469-e8e7-4d90-8ee0-00d054642a94 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:09.592678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7944469-e8e7-4d90-8ee0-00d054642a94 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:09.597308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d744895-a065-4668-8d0d-d0f3766136ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:09.597922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d744895-a065-4668-8d0d-d0f3766136ab admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.598231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d744895-a065-4668-8d0d-d0f3766136ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.598503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d744895-a065-4668-8d0d-d0f3766136ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.599134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d744895-a065-4668-8d0d-d0f3766136ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:09.601013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d744895-a065-4668-8d0d-d0f3766136ab admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:09.615917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:09.666707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1020/2071] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:09 2022] POST /identity/v3/projects => generated 391 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:09.670830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:09.681763 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac446ec-ac7d-4010-bcf6-4967a8b289cd admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:09.736743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acd0de-a679-4dae-a125-a6974aea1a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:09.737338 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acd0de-a679-4dae-a125-a6974aea1a76 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.737617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acd0de-a679-4dae-a125-a6974aea1a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.737886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acd0de-a679-4dae-a125-a6974aea1a76 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.738548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acd0de-a679-4dae-a125-a6974aea1a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:09.740428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acd0de-a679-4dae-a125-a6974aea1a76 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:09.797680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1052/2072] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:09 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:09.801016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d744895-a065-4668-8d0d-d0f3766136ab admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:09.812030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d744895-a065-4668-8d0d-d0f3766136ab admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:09.816662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186acb-ae85-4cff-b89a-12e09252cb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:09.817256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186acb-ae85-4cff-b89a-12e09252cb15 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.817536 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186acb-ae85-4cff-b89a-12e09252cb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.817825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186acb-ae85-4cff-b89a-12e09252cb15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.818609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4186acb-ae85-4cff-b89a-12e09252cb15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:09.820232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4186acb-ae85-4cff-b89a-12e09252cb15 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:09.830044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1021/2073] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:09.833869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acd0de-a679-4dae-a125-a6974aea1a76 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:09.844233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acd0de-a679-4dae-a125-a6974aea1a76 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:09.849090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6aaa6-e840-40e4-96b7-afb05a04c913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:09.849688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6aaa6-e840-40e4-96b7-afb05a04c913 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.849994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6aaa6-e840-40e4-96b7-afb05a04c913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.850272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6aaa6-e840-40e4-96b7-afb05a04c913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.850927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6aaa6-e840-40e4-96b7-afb05a04c913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:09.853131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6aaa6-e840-40e4-96b7-afb05a04c913 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:09.864232 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1053/2074] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:43:09.872983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186acb-ae85-4cff-b89a-12e09252cb15 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:09.882906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186acb-ae85-4cff-b89a-12e09252cb15 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:09.887494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:09.888141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.888417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.888693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] PATH_INFO: `/v3/projects/f272a57129dc428d9c45e8ae308fa026/users/9edb3f724cc142ee843834352b9a3b9f/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.903622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f272a57129dc428d9c45e8ae308fa026, user_id=9edb3f724cc142ee843834352b9a3b9f, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:09.906311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:09.956390 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1022/2075] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:09 2022] PUT /identity/v3/projects/f272a57129dc428d9c45e8ae308fa026/users/9edb3f724cc142ee843834352b9a3b9f/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:09.967521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d8d5d-f6d4-4fef-bb04-03a3b5d69762 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:09.967800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d8d5d-f6d4-4fef-bb04-03a3b5d69762 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:09.968075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d8d5d-f6d4-4fef-bb04-03a3b5d69762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:09.996258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-fd9d8d5d-f6d4-4fef-bb04-03a3b5d69762 None None] MFA Rules not processed for user `9edb3f724cc142ee843834352b9a3b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:09.997822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fd9d8d5d-f6d4-4fef-bb04-03a3b5d69762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:10.010789 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:10.062974 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1054/2076] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:43:10.073732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:10.075457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:10.132730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c487ca3-75c1-41f6-be28-5c44d6d356d5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:10.136344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:10.136947 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:10.137227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:10.137501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:10.145804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:10.147249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:10.156448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1023/2077] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:12.830958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d8d5d-f6d4-4fef-bb04-03a3b5d69762 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:12.832424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-fd9d8d5d-f6d4-4fef-bb04-03a3b5d69762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:12.906730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d8d5d-f6d4-4fef-bb04-03a3b5d69762 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:12.912223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:12.912824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:12.913101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:12.913370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:12.915145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:12.972734 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:12.974365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:12.983211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1055/2078] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:16.913389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:16.914848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:16.925969 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366742ae-0804-4e5d-bde1-e06777ce80ae service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:16.929552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:16.930175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:16.930952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:16.931232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:16.932878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:16.992439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:16.994396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:17.003504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1024/2079] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:19.402513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.413493 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcff4a-d0aa-46c0-816d-5714ada9e6ab service nova] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.419669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409866b-022d-4b2e-bd64-6ee1d7f1c347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.420288 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409866b-022d-4b2e-bd64-6ee1d7f1c347 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.420562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409866b-022d-4b2e-bd64-6ee1d7f1c347 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.420829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409866b-022d-4b2e-bd64-6ee1d7f1c347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.422542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d409866b-022d-4b2e-bd64-6ee1d7f1c347 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:19.470879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:19.481302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409866b-022d-4b2e-bd64-6ee1d7f1c347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.483133 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409866b-022d-4b2e-bd64-6ee1d7f1c347 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.492507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1056/2080] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:26.999391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=2,keystone:SELECT=71 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:29.493788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:30.644910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.707885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db75ab-e085-4881-a0f7-866d9d561c88 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.712426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.713058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.713339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.713616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] PATH_INFO: `/v3/users/9edb3f724cc142ee843834352b9a3b9f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.714777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9edb3f724cc142ee843834352b9a3b9f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.717216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.886299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9edb3f724cc142ee843834352b9a3b9f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:30.900613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9edb3f724cc142ee843834352b9a3b9f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:30.945801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9edb3f724cc142ee843834352b9a3b9f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:30.946545 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9edb3f724cc142ee843834352b9a3b9f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:30.955805 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9edb3f724cc142ee843834352b9a3b9f'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:30.962421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1025/2081] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:30 2022] DELETE /identity/v3/users/9edb3f724cc142ee843834352b9a3b9f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:30.967606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fa925-51ac-43c4-ab72-d6d4abbba7a7 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.968106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fa925-51ac-43c4-ab72-d6d4abbba7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.968216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fa925-51ac-43c4-ab72-d6d4abbba7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.989536 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-506fa925-51ac-43c4-ab72-d6d4abbba7a7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:30.990813 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-506fa925-51ac-43c4-ab72-d6d4abbba7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:31.050310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1057/2082] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:31.060071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:31.062176 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:31.118951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eb20f-1611-48ae-b255-8be661f69b27 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:31.123565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee1b2c-cf02-4f06-9694-1f1ac4db1484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:31.124160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee1b2c-cf02-4f06-9694-1f1ac4db1484 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:31.124440 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee1b2c-cf02-4f06-9694-1f1ac4db1484 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:31.124702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee1b2c-cf02-4f06-9694-1f1ac4db1484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:31.133511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee1b2c-cf02-4f06-9694-1f1ac4db1484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:31.134963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee1b2c-cf02-4f06-9694-1f1ac4db1484 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:31.144499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1026/2083] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:31.616164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fa925-51ac-43c4-ab72-d6d4abbba7a7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:31.617602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-506fa925-51ac-43c4-ab72-d6d4abbba7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:31.630232 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fa925-51ac-43c4-ab72-d6d4abbba7a7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:31.635110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:31.635789 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:31.635993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:31.636253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] PATH_INFO: `/v3/projects/f272a57129dc428d9c45e8ae308fa026` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:31.637368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f272a57129dc428d9c45e8ae308fa026)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:31.639857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:31.752579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f272a57129dc428d9c45e8ae308fa026', 'request_id': 'req-43edce6e-b16d-4986-894d-667f0eb3b974'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:31.758329 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f272a57129dc428d9c45e8ae308fa026', 'request_id': 'req-43edce6e-b16d-4986-894d-667f0eb3b974'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:31.765126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f272a57129dc428d9c45e8ae308fa026', 'request_id': 'req-43edce6e-b16d-4986-894d-667f0eb3b974'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:31.836450 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] The token cache is being invalidate because project f272a57129dc428d9c45e8ae308fa026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:31.836743 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:31.837441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-43edce6e-b16d-4986-894d-667f0eb3b974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:31.838976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1058/2084] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:31 2022] DELETE /identity/v3/projects/f272a57129dc428d9c45e8ae308fa026 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:31.844756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79810-7756-46d0-b622-26bf891eb63a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:31.845262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79810-7756-46d0-b622-26bf891eb63a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:31.845365 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b79810-7756-46d0-b622-26bf891eb63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:31.864827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-97b79810-7756-46d0-b622-26bf891eb63a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:31.866107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-97b79810-7756-46d0-b622-26bf891eb63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:31.882725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:31.924164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1027/2085] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:31.988850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e080b5-bbe7-40d2-9d54-a4edee12148e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:31.989375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e080b5-bbe7-40d2-9d54-a4edee12148e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:31.989375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e080b5-bbe7-40d2-9d54-a4edee12148e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.009800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-13e080b5-bbe7-40d2-9d54-a4edee12148e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:32.011027 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-13e080b5-bbe7-40d2-9d54-a4edee12148e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:32.019555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1059/2086] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:32.023927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79810-7756-46d0-b622-26bf891eb63a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:32.033945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b79810-7756-46d0-b622-26bf891eb63a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:32.038659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:32.039263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:32.039546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:32.039821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.040468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:32.040773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:32.042430 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:32.053844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1028/2087] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:32.057922 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e080b5-bbe7-40d2-9d54-a4edee12148e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:32.067761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e080b5-bbe7-40d2-9d54-a4edee12148e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:32.073152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d774-25de-4b37-ab0c-4dd4d1b1bf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:32.073894 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d774-25de-4b37-ab0c-4dd4d1b1bf45 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:32.074200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d774-25de-4b37-ab0c-4dd4d1b1bf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:32.074468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d774-25de-4b37-ab0c-4dd4d1b1bf45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.075089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6d774-25de-4b37-ab0c-4dd4d1b1bf45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:32.077695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6d774-25de-4b37-ab0c-4dd4d1b1bf45 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:32.169787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1060/2088] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:32 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:32.174292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:32.184249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb6710-af1b-4aed-a957-3389a49b7bd7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:32.241052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0cdc0-a123-46bb-9338-4ce581dd17bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:32.241604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0cdc0-a123-46bb-9338-4ce581dd17bb admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:32.241879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0cdc0-a123-46bb-9338-4ce581dd17bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:32.242178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0cdc0-a123-46bb-9338-4ce581dd17bb admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.242864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0cdc0-a123-46bb-9338-4ce581dd17bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:32.244754 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0cdc0-a123-46bb-9338-4ce581dd17bb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:32.340553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1029/2089] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:32 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:32.344835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d774-25de-4b37-ab0c-4dd4d1b1bf45 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:32.355021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d774-25de-4b37-ab0c-4dd4d1b1bf45 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:32.360412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99a13-0bfc-43d2-8d3e-a1af084f918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:32.361044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99a13-0bfc-43d2-8d3e-a1af084f918f admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:32.361342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99a13-0bfc-43d2-8d3e-a1af084f918f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:32.361616 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99a13-0bfc-43d2-8d3e-a1af084f918f admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.362303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e99a13-0bfc-43d2-8d3e-a1af084f918f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:32.363899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e99a13-0bfc-43d2-8d3e-a1af084f918f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:32.376039 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1061/2090] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:43:32.386549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0cdc0-a123-46bb-9338-4ce581dd17bb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:32.396701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0cdc0-a123-46bb-9338-4ce581dd17bb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:32.401436 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a011-b64f-4faf-ae01-36777e3d073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:32.402069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a011-b64f-4faf-ae01-36777e3d073b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:32.402350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a011-b64f-4faf-ae01-36777e3d073b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:32.402618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a011-b64f-4faf-ae01-36777e3d073b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.403278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9a011-b64f-4faf-ae01-36777e3d073b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:32.404855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9a011-b64f-4faf-ae01-36777e3d073b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:32.415733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1030/2091] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:32.420437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99a13-0bfc-43d2-8d3e-a1af084f918f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:32.430278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99a13-0bfc-43d2-8d3e-a1af084f918f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:32.434898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:32.436218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:32.436479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:32.436754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] PATH_INFO: `/v3/projects/00c9aab9a0264449ac51194afd4cb6a9/users/205b19ccb80a45fd8ed36abd2c4c32c3/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.454402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=00c9aab9a0264449ac51194afd4cb6a9, user_id=205b19ccb80a45fd8ed36abd2c4c32c3, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:32.454715 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:32.535061 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1062/2092] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:32 2022] PUT /identity/v3/projects/00c9aab9a0264449ac51194afd4cb6a9/users/205b19ccb80a45fd8ed36abd2c4c32c3/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:32.540194 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ead31-d579-4bc4-9086-99d0098bdfd5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:32.540477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ead31-d579-4bc4-9086-99d0098bdfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:32.540770 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ead31-d579-4bc4-9086-99d0098bdfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.560374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-090ead31-d579-4bc4-9086-99d0098bdfd5 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:32.562222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-090ead31-d579-4bc4-9086-99d0098bdfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:32.621240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1031/2093] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:32.631592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:32.633292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:32.656009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:32.690913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbf5b2-38a5-4760-8a3b-14c5f96cde2f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:32.694504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039e316-f6c9-4003-a8e0-d0b231914c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:32.695093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039e316-f6c9-4003-a8e0-d0b231914c1d service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:32.695368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039e316-f6c9-4003-a8e0-d0b231914c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:32.695634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039e316-f6c9-4003-a8e0-d0b231914c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:32.704476 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039e316-f6c9-4003-a8e0-d0b231914c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:32.710224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039e316-f6c9-4003-a8e0-d0b231914c1d service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:32.715486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1063/2094] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:37.527835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ead31-d579-4bc4-9086-99d0098bdfd5 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:37.529327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-090ead31-d579-4bc4-9086-99d0098bdfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:37.581155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:37.647677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ead31-d579-4bc4-9086-99d0098bdfd5 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:37.656188 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:37.656840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:37.657104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:37.657379 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:37.659530 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:37.674027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:37.675643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:37.712712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1032/2095] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:38.950503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25432f62-bfe8-492c-bdc8-4930b521ab97 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.951050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25432f62-bfe8-492c-bdc8-4930b521ab97 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.951050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25432f62-bfe8-492c-bdc8-4930b521ab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.984975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-25432f62-bfe8-492c-bdc8-4930b521ab97 None None] MFA Rules not processed for user `205b19ccb80a45fd8ed36abd2c4c32c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:38.986242 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-25432f62-bfe8-492c-bdc8-4930b521ab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:39.055348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1064/2096] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:39.059841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.071655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a97a0-d39f-44ce-852f-04f9a4a81220 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.077966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952640c-0781-4777-bed0-cdd2c0820a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.078580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952640c-0781-4777-bed0-cdd2c0820a6a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.078855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952640c-0781-4777-bed0-cdd2c0820a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.079124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952640c-0781-4777-bed0-cdd2c0820a6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.079776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952640c-0781-4777-bed0-cdd2c0820a6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.081658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952640c-0781-4777-bed0-cdd2c0820a6a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.199730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1033/2097] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:39 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:39.204579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25432f62-bfe8-492c-bdc8-4930b521ab97 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.216309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25432f62-bfe8-492c-bdc8-4930b521ab97 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.275293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dec55b-2c94-44e1-8aa8-cd4e18b09ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.275953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dec55b-2c94-44e1-8aa8-cd4e18b09ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.276241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dec55b-2c94-44e1-8aa8-cd4e18b09ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.277129 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dec55b-2c94-44e1-8aa8-cd4e18b09ee2 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.277824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dec55b-2c94-44e1-8aa8-cd4e18b09ee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.279815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dec55b-2c94-44e1-8aa8-cd4e18b09ee2 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.363334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1065/2098] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:39 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:39.367323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952640c-0781-4777-bed0-cdd2c0820a6a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.378623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952640c-0781-4777-bed0-cdd2c0820a6a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.383433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d1eb3-63da-4519-b24d-30a2375a051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.384067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d1eb3-63da-4519-b24d-30a2375a051c admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.384351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d1eb3-63da-4519-b24d-30a2375a051c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.388926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d1eb3-63da-4519-b24d-30a2375a051c admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.388926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d1eb3-63da-4519-b24d-30a2375a051c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.388926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d1eb3-63da-4519-b24d-30a2375a051c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.401335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1034/2099] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:39.402631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dec55b-2c94-44e1-8aa8-cd4e18b09ee2 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.413171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dec55b-2c94-44e1-8aa8-cd4e18b09ee2 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.417782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc90b9c-1fa7-4467-b2e0-f4bbdeae8895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.418459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc90b9c-1fa7-4467-b2e0-f4bbdeae8895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.418740 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc90b9c-1fa7-4467-b2e0-f4bbdeae8895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.419009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc90b9c-1fa7-4467-b2e0-f4bbdeae8895 admin admin] PATH_INFO: `/v3/projects/59c7b59608464ffdbe9f70ad618aa044/users/f5c74c3045d6458ea02bf89e3d62d18a/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.434109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc90b9c-1fa7-4467-b2e0-f4bbdeae8895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c7b59608464ffdbe9f70ad618aa044, user_id=f5c74c3045d6458ea02bf89e3d62d18a, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.437812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc90b9c-1fa7-4467-b2e0-f4bbdeae8895 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.527415 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1066/2100] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:39 2022] PUT /identity/v3/projects/59c7b59608464ffdbe9f70ad618aa044/users/f5c74c3045d6458ea02bf89e3d62d18a/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:39.528656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d1eb3-63da-4519-b24d-30a2375a051c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.539876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d1eb3-63da-4519-b24d-30a2375a051c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.611127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d71ad-b525-469d-ae1c-8f4aa78f1778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.611732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d71ad-b525-469d-ae1c-8f4aa78f1778 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.612010 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d71ad-b525-469d-ae1c-8f4aa78f1778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.613167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d71ad-b525-469d-ae1c-8f4aa78f1778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.613167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d71ad-b525-469d-ae1c-8f4aa78f1778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.614511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d71ad-b525-469d-ae1c-8f4aa78f1778 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.628516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1035/2101] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:39.629700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc90b9c-1fa7-4467-b2e0-f4bbdeae8895 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.640757 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc90b9c-1fa7-4467-b2e0-f4bbdeae8895 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.645347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ac309-f75f-495c-9d89-c1ef5621b317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.646042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac309-f75f-495c-9d89-c1ef5621b317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.646319 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac309-f75f-495c-9d89-c1ef5621b317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.646590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac309-f75f-495c-9d89-c1ef5621b317 admin admin] PATH_INFO: `/v3/projects/59c7b59608464ffdbe9f70ad618aa044/users/f5c74c3045d6458ea02bf89e3d62d18a/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.648565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ac309-f75f-495c-9d89-c1ef5621b317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c7b59608464ffdbe9f70ad618aa044, user_id=f5c74c3045d6458ea02bf89e3d62d18a, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.651259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ac309-f75f-495c-9d89-c1ef5621b317 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.662105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1067/2102] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:39 2022] PUT /identity/v3/projects/59c7b59608464ffdbe9f70ad618aa044/users/f5c74c3045d6458ea02bf89e3d62d18a/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:45.992639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d71ad-b525-469d-ae1c-8f4aa78f1778 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:46.149570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d71ad-b525-469d-ae1c-8f4aa78f1778 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:46.178931 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:46.244544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9c7ec-d236-49a6-95a5-07a9065106c9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:46.249989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9c7ec-d236-49a6-95a5-07a9065106c9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:46.250281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9c7ec-d236-49a6-95a5-07a9065106c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:46.271452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:46.310633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-10b9c7ec-d236-49a6-95a5-07a9065106c9 None None] MFA Rules not processed for user `f5c74c3045d6458ea02bf89e3d62d18a`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:46.312931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-10b9c7ec-d236-49a6-95a5-07a9065106c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:46.334380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:46.335321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:46.336401 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:46.336852 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:46.341894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:46.376317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:46.379043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:46.426499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1036/2103] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:46.473587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1068/2104] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:46.491618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:46.493173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:46.511190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a806c-75c2-460d-8830-cd9ec787664e service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:46.518542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e9bf3-81ec-40be-b40e-c7e0cb7bbb18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:46.519311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e9bf3-81ec-40be-b40e-c7e0cb7bbb18 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:46.519817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e9bf3-81ec-40be-b40e-c7e0cb7bbb18 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:46.520125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e9bf3-81ec-40be-b40e-c7e0cb7bbb18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:46.590568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e9bf3-81ec-40be-b40e-c7e0cb7bbb18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:46.592195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e9bf3-81ec-40be-b40e-c7e0cb7bbb18 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:46.602394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1037/2105] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:48.013304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9c7ec-d236-49a6-95a5-07a9065106c9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:48.117121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9c7ec-d236-49a6-95a5-07a9065106c9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:48.121745 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:48.122578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:48.123210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:48.123569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:48.126480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:48.203265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:48.204833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:48.220647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1069/2106] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:51.686681 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1d0d0-7d44-4768-bdca-436af913c65a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:51.687234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1d0d0-7d44-4768-bdca-436af913c65a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:51.687234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1d0d0-7d44-4768-bdca-436af913c65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:51.707308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-12e1d0d0-7d44-4768-bdca-436af913c65a None None] MFA Rules not processed for user `39543aedd1834f68a9d0a4ef174822c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:51.708520 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-12e1d0d0-7d44-4768-bdca-436af913c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:51.766082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1038/2107] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:43:52.132087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:52.133674 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:52.144661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbba27d-7adf-4cfe-b968-b38131e44e4b service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:52.148549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:52.149245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.149584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.149921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.151731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:52.164039 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:52.165723 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:52.175739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1070/2108] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:52.792110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1d0d0-7d44-4768-bdca-436af913c65a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:52.792110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-12e1d0d0-7d44-4768-bdca-436af913c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:52.893080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:52.926678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1d0d0-7d44-4768-bdca-436af913c65a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:52.943886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:52.944743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.945183 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.945599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.947739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:52.963871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:52.965884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:52.993026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1039/2109] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:53.268206 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:53.270277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:53.287821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32278e-d159-4322-9141-93e8401266b0 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:53.292885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:53.293549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:53.293829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:53.294172 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] PATH_INFO: `/v3/users/19d863a45ac14c50b940fbc217b70929` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:53.296041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d863a45ac14c50b940fbc217b70929)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:53.298169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:53.322322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:53.494714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d863a45ac14c50b940fbc217b70929'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:53.517304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d863a45ac14c50b940fbc217b70929'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:53.518062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d863a45ac14c50b940fbc217b70929'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:53.621365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d863a45ac14c50b940fbc217b70929'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:53.629720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-904c793f-3e2f-45b0-8bd3-ebcec43248c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d863a45ac14c50b940fbc217b70929'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:53.635938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1071/2110] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:53 2022] DELETE /identity/v3/users/19d863a45ac14c50b940fbc217b70929 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:54.234723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:54.248696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae8801-bdf0-475a-ad30-d99889c307ad service placement] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:54.327845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:54.328691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:54.328989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:54.329228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] PATH_INFO: `/v3/projects/d62398af777d4d269977b1a5f1f37080` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:54.330476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62398af777d4d269977b1a5f1f37080)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:54.332485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:54.486288 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62398af777d4d269977b1a5f1f37080', 'request_id': 'req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:54.492071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62398af777d4d269977b1a5f1f37080', 'request_id': 'req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:54.499398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62398af777d4d269977b1a5f1f37080', 'request_id': 'req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:54.605711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] The token cache is being invalidate because project d62398af777d4d269977b1a5f1f37080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:54.606033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:54.606687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d368f3ee-b7ba-4c43-b2da-3d9987e02f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:54.608775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1040/2111] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:54 2022] DELETE /identity/v3/projects/d62398af777d4d269977b1a5f1f37080 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:54.613770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5706c4-0871-4a6d-9de4-a211032b0f91 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:54.614263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5706c4-0871-4a6d-9de4-a211032b0f91 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:54.614378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5706c4-0871-4a6d-9de4-a211032b0f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:54.648710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-cc5706c4-0871-4a6d-9de4-a211032b0f91 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:54.648710 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-cc5706c4-0871-4a6d-9de4-a211032b0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:54.703025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1072/2112] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:54.710664 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c42aa-47ba-48ce-b6e5-cd0ec7dacaf0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:54.711801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c42aa-47ba-48ce-b6e5-cd0ec7dacaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:54.711801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c42aa-47ba-48ce-b6e5-cd0ec7dacaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:54.732418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ed3c42aa-47ba-48ce-b6e5-cd0ec7dacaf0 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:54.734241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ed3c42aa-47ba-48ce-b6e5-cd0ec7dacaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:54.742823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1041/2113] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:54.746987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5706c4-0871-4a6d-9de4-a211032b0f91 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:54.757056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5706c4-0871-4a6d-9de4-a211032b0f91 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:54.762480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:54.763084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:54.763368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:54.763747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:54.764569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:54.764860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:54.766544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:54.783445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1073/2114] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:54.784592 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c42aa-47ba-48ce-b6e5-cd0ec7dacaf0 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:54.795003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c42aa-47ba-48ce-b6e5-cd0ec7dacaf0 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:54.800007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f861d99-bf87-4b7c-847e-c2e594f68eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:54.800751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f861d99-bf87-4b7c-847e-c2e594f68eeb admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:54.801036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f861d99-bf87-4b7c-847e-c2e594f68eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:54.801311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f861d99-bf87-4b7c-847e-c2e594f68eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:54.801989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f861d99-bf87-4b7c-847e-c2e594f68eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:54.803977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f861d99-bf87-4b7c-847e-c2e594f68eeb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:54.891776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1042/2115] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:54 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:54.895883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:54.905986 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18283f2e-57ae-4eb7-b8bd-1abc44b4109a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:54.988195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b632563-0b31-4198-bd46-62ed5af552cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:54.988800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632563-0b31-4198-bd46-62ed5af552cf admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:54.989084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632563-0b31-4198-bd46-62ed5af552cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:54.989350 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632563-0b31-4198-bd46-62ed5af552cf admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:54.990023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b632563-0b31-4198-bd46-62ed5af552cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:54.992058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b632563-0b31-4198-bd46-62ed5af552cf admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:55.099304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1074/2116] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:54 2022] POST /identity/v3/users => generated 303 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:55.100787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f861d99-bf87-4b7c-847e-c2e594f68eeb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:55.112802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f861d99-bf87-4b7c-847e-c2e594f68eeb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:55.119661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534e5a-bf5d-442b-97e3-03a94c071e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:55.119837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534e5a-bf5d-442b-97e3-03a94c071e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:55.120124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534e5a-bf5d-442b-97e3-03a94c071e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:55.120462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39534e5a-bf5d-442b-97e3-03a94c071e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:55.121191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39534e5a-bf5d-442b-97e3-03a94c071e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:55.124294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39534e5a-bf5d-442b-97e3-03a94c071e6a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:55.139659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1043/2117] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:43:55.144128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b632563-0b31-4198-bd46-62ed5af552cf admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:55.155334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b632563-0b31-4198-bd46-62ed5af552cf admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:55.160034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc1aa7-95dc-441a-81dd-279e3635396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:55.161412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc1aa7-95dc-441a-81dd-279e3635396a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:55.161704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc1aa7-95dc-441a-81dd-279e3635396a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:55.162347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc1aa7-95dc-441a-81dd-279e3635396a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:55.162991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc1aa7-95dc-441a-81dd-279e3635396a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:55.164580 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc1aa7-95dc-441a-81dd-279e3635396a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:55.176583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1075/2118] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:55.180691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534e5a-bf5d-442b-97e3-03a94c071e6a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:55.191623 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39534e5a-bf5d-442b-97e3-03a94c071e6a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:55.196566 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a580296-d193-44ae-97a8-bbb09acc3f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:55.197385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a580296-d193-44ae-97a8-bbb09acc3f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:55.203889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a580296-d193-44ae-97a8-bbb09acc3f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:55.203889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a580296-d193-44ae-97a8-bbb09acc3f68 admin admin] PATH_INFO: `/v3/projects/3620332e24494ea3b591934c3f77afc0/users/d59b4abcd4fa4ba9bcee86cddefdbb4b/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:55.214972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a580296-d193-44ae-97a8-bbb09acc3f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3620332e24494ea3b591934c3f77afc0, user_id=d59b4abcd4fa4ba9bcee86cddefdbb4b, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:55.218185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a580296-d193-44ae-97a8-bbb09acc3f68 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:55.280856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:55.285810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1044/2119] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:55 2022] PUT /identity/v3/projects/3620332e24494ea3b591934c3f77afc0/users/d59b4abcd4fa4ba9bcee86cddefdbb4b/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:55.297903 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed1b53-1531-418f-9bcf-8c6d08607312 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:55.298228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed1b53-1531-418f-9bcf-8c6d08607312 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:55.298505 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed1b53-1531-418f-9bcf-8c6d08607312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:55.306576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:55.326924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-38ed1b53-1531-418f-9bcf-8c6d08607312 None None] MFA Rules not processed for user `d59b4abcd4fa4ba9bcee86cddefdbb4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:55.328871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-38ed1b53-1531-418f-9bcf-8c6d08607312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:55.409147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1076/2120] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:55.413633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a580296-d193-44ae-97a8-bbb09acc3f68 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:55.484469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a580296-d193-44ae-97a8-bbb09acc3f68 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:55.492605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8e660-3abc-4da5-b2ab-4c4c31c00da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:55.493318 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8e660-3abc-4da5-b2ab-4c4c31c00da6 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:55.493741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8e660-3abc-4da5-b2ab-4c4c31c00da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:55.494116 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8e660-3abc-4da5-b2ab-4c4c31c00da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:55.494849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8e660-3abc-4da5-b2ab-4c4c31c00da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:55.496878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8e660-3abc-4da5-b2ab-4c4c31c00da6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:55.591154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1045/2121] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:55 2022] POST /identity/v3/projects => generated 405 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:55.593198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed1b53-1531-418f-9bcf-8c6d08607312 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:55.605549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed1b53-1531-418f-9bcf-8c6d08607312 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:55.674822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694fc5-d7d3-46e8-9713-7f666c90c8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:55.675496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694fc5-d7d3-46e8-9713-7f666c90c8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:55.675988 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694fc5-d7d3-46e8-9713-7f666c90c8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:55.676334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694fc5-d7d3-46e8-9713-7f666c90c8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:55.677066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7694fc5-d7d3-46e8-9713-7f666c90c8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:55.679209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7694fc5-d7d3-46e8-9713-7f666c90c8f0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:55.837639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1077/2122] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:55 2022] POST /identity/v3/users => generated 309 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:55.839136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8e660-3abc-4da5-b2ab-4c4c31c00da6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:55.851289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8e660-3abc-4da5-b2ab-4c4c31c00da6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:55.855986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927370d6-54f9-442f-902b-e173e0bebcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:55.856672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927370d6-54f9-442f-902b-e173e0bebcfd admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:55.856953 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927370d6-54f9-442f-902b-e173e0bebcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:55.857221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927370d6-54f9-442f-902b-e173e0bebcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:55.858008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927370d6-54f9-442f-902b-e173e0bebcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:55.859682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927370d6-54f9-442f-902b-e173e0bebcfd admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:55.871220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1046/2123] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:55.875682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694fc5-d7d3-46e8-9713-7f666c90c8f0 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:55.894849 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694fc5-d7d3-46e8-9713-7f666c90c8f0 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:55.899700 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0d18a-d45e-4436-9800-dcdb430ccfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:55.900369 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0d18a-d45e-4436-9800-dcdb430ccfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:55.901045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0d18a-d45e-4436-9800-dcdb430ccfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:55.901045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0d18a-d45e-4436-9800-dcdb430ccfdb admin admin] PATH_INFO: `/v3/projects/f353b716a8824b5683b8d2d453f65f45/users/18b84ea5e2cb4adaa961dfa6aa0f3486/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:55.921182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0d18a-d45e-4436-9800-dcdb430ccfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f353b716a8824b5683b8d2d453f65f45, user_id=18b84ea5e2cb4adaa961dfa6aa0f3486, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:55.924019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0d18a-d45e-4436-9800-dcdb430ccfdb admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:55.999997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1078/2124] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:55 2022] PUT /identity/v3/projects/f353b716a8824b5683b8d2d453f65f45/users/18b84ea5e2cb4adaa961dfa6aa0f3486/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:56.004114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927370d6-54f9-442f-902b-e173e0bebcfd admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:56.013988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927370d6-54f9-442f-902b-e173e0bebcfd admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:56.078032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c62f8-dda6-499f-945f-dda248ff53fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:56.078672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c62f8-dda6-499f-945f-dda248ff53fb admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:56.078948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c62f8-dda6-499f-945f-dda248ff53fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:56.079219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c62f8-dda6-499f-945f-dda248ff53fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:56.079858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c62f8-dda6-499f-945f-dda248ff53fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:56.081431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c62f8-dda6-499f-945f-dda248ff53fb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:56.093068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1047/2125] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:56.097099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0d18a-d45e-4436-9800-dcdb430ccfdb admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:56.109525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0d18a-d45e-4436-9800-dcdb430ccfdb admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:56.118439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:56.119353 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:56.119812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:56.120247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] PATH_INFO: `/v3/projects/f353b716a8824b5683b8d2d453f65f45/users/18b84ea5e2cb4adaa961dfa6aa0f3486/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:56.122309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f353b716a8824b5683b8d2d453f65f45, user_id=18b84ea5e2cb4adaa961dfa6aa0f3486, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:56.125448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:56.137314 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1079/2126] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:56 2022] PUT /identity/v3/projects/f353b716a8824b5683b8d2d453f65f45/users/18b84ea5e2cb4adaa961dfa6aa0f3486/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:56.150127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3dfcc-5b35-400d-b1f9-fba782ea1363 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:56.150127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3dfcc-5b35-400d-b1f9-fba782ea1363 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:56.150127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3dfcc-5b35-400d-b1f9-fba782ea1363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:56.192368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b1a3dfcc-5b35-400d-b1f9-fba782ea1363 None None] MFA Rules not processed for user `18b84ea5e2cb4adaa961dfa6aa0f3486`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:56.193653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b1a3dfcc-5b35-400d-b1f9-fba782ea1363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:56.274182 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:56.282501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1048/2127] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:56.292610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:56.296114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:56.368666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:56.379550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46e2b0-3c33-4a9f-bc88-564c928831e5 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:56.384529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:56.385183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:56.385464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:56.385733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:56.395889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:56.397596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:56.410092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1080/2128] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:56 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:56.585262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3dfcc-5b35-400d-b1f9-fba782ea1363 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:56.667192 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3dfcc-5b35-400d-b1f9-fba782ea1363 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:56.675893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:56.680245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:56.680245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:56.680245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] PATH_INFO: `/v3/users/d59b4abcd4fa4ba9bcee86cddefdbb4b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:56.680245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d59b4abcd4fa4ba9bcee86cddefdbb4b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:56.680768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:56.770938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:56.772355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:56.832403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c59619-4cec-4f0d-b37c-d26896eee7e2 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:56.836222 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2897dd-e6eb-431b-8966-da487a695652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:56.836771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2897dd-e6eb-431b-8966-da487a695652 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:56.837041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2897dd-e6eb-431b-8966-da487a695652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:56.837307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2897dd-e6eb-431b-8966-da487a695652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:56.842728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ec2897dd-e6eb-431b-8966-da487a695652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:56.853651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2897dd-e6eb-431b-8966-da487a695652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:56.855211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2897dd-e6eb-431b-8966-da487a695652 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:56.865164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1081/2129] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:43:56.883088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd59b4abcd4fa4ba9bcee86cddefdbb4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:56.898051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd59b4abcd4fa4ba9bcee86cddefdbb4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:56.954325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd59b4abcd4fa4ba9bcee86cddefdbb4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:56.954325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd59b4abcd4fa4ba9bcee86cddefdbb4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:56.964447 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd59b4abcd4fa4ba9bcee86cddefdbb4b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:56.971508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1049/2130] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:56 2022] DELETE /identity/v3/users/d59b4abcd4fa4ba9bcee86cddefdbb4b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:56.976001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d0c8c-b1c7-435d-b702-2452817c2ba8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:56.976283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d0c8c-b1c7-435d-b702-2452817c2ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:56.976553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d0c8c-b1c7-435d-b702-2452817c2ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:56.996507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-a98d0c8c-b1c7-435d-b702-2452817c2ba8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:56.997780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a98d0c8c-b1c7-435d-b702-2452817c2ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:57.062627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1082/2131] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:43:57.077686 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:57.079213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:57.144957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dffbd-56fe-4ad2-9901-82d873438d3e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:57.148643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1308d1a-aa1a-480c-a089-d12ef701f51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:57.149249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1308d1a-aa1a-480c-a089-d12ef701f51c service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:57.149523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1308d1a-aa1a-480c-a089-d12ef701f51c service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:57.149796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1308d1a-aa1a-480c-a089-d12ef701f51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:57.159695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1308d1a-aa1a-480c-a089-d12ef701f51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:57.161225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1308d1a-aa1a-480c-a089-d12ef701f51c service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:57.172160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1050/2132] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:43:57.676445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d0c8c-b1c7-435d-b702-2452817c2ba8 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:57.678617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a98d0c8c-b1c7-435d-b702-2452817c2ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:57.697507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d0c8c-b1c7-435d-b702-2452817c2ba8 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:57.703551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:57.704265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:57.705999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:57.705999 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] PATH_INFO: `/v3/projects/3620332e24494ea3b591934c3f77afc0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:57.712271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3620332e24494ea3b591934c3f77afc0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:57.712271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:57.872422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3620332e24494ea3b591934c3f77afc0', 'request_id': 'req-509ac9b0-dabb-4b0b-8023-1d0959790325'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:57.892558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3620332e24494ea3b591934c3f77afc0', 'request_id': 'req-509ac9b0-dabb-4b0b-8023-1d0959790325'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:57.913253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3620332e24494ea3b591934c3f77afc0', 'request_id': 'req-509ac9b0-dabb-4b0b-8023-1d0959790325'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:57.992339 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] The token cache is being invalidate because project 3620332e24494ea3b591934c3f77afc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:57.992554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:57.993131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:57.994649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1083/2133] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:57 2022] DELETE /identity/v3/projects/3620332e24494ea3b591934c3f77afc0 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:58.000561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1308d1a-aa1a-480c-a089-d12ef701f51c service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:58.013571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1308d1a-aa1a-480c-a089-d12ef701f51c service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:58.026479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:58.073990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:58.074210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:58.074487 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:58.074759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] PATH_INFO: `/v3/users/18b84ea5e2cb4adaa961dfa6aa0f3486` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:58.075935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b84ea5e2cb4adaa961dfa6aa0f3486)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:58.078501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:58.265052 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b84ea5e2cb4adaa961dfa6aa0f3486'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:58.277540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b84ea5e2cb4adaa961dfa6aa0f3486'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:58.358203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b84ea5e2cb4adaa961dfa6aa0f3486'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:58.358841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b84ea5e2cb4adaa961dfa6aa0f3486'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:58.366790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f08332f5-8006-4a90-b9d3-71419d36c021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b84ea5e2cb4adaa961dfa6aa0f3486'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:58.373104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1051/2134] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:57 2022] DELETE /identity/v3/users/18b84ea5e2cb4adaa961dfa6aa0f3486 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:58.902545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:58.904145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:58.925452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ac9b0-dabb-4b0b-8023-1d0959790325 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:58.942712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:58.989706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:58.995842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:58.995842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:58.995842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] PATH_INFO: `/v3/projects/f353b716a8824b5683b8d2d453f65f45` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:58.996944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f353b716a8824b5683b8d2d453f65f45)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:58.999351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:59.125220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f353b716a8824b5683b8d2d453f65f45', 'request_id': 'req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:59.132644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f353b716a8824b5683b8d2d453f65f45', 'request_id': 'req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:59.139693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f353b716a8824b5683b8d2d453f65f45', 'request_id': 'req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:59.203677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] The token cache is being invalidate because project f353b716a8824b5683b8d2d453f65f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:59.209187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:59.209187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2c3ee327-8e19-4cf4-a07f-50a9f2b9d1b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:59.209187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1084/2135] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:58 2022] DELETE /identity/v3/projects/f353b716a8824b5683b8d2d453f65f45 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:59.211049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4ad7c-eb1b-48fb-8d06-3ca5a40c26fe None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.211476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4ad7c-eb1b-48fb-8d06-3ca5a40c26fe None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.211594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4ad7c-eb1b-48fb-8d06-3ca5a40c26fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.231857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-96b4ad7c-eb1b-48fb-8d06-3ca5a40c26fe None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:59.233730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-96b4ad7c-eb1b-48fb-8d06-3ca5a40c26fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:59.291923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1052/2136] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:59.300433 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a59742-cf55-46fe-bcf9-4d1e60d69c78 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.300879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a59742-cf55-46fe-bcf9-4d1e60d69c78 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.301268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a59742-cf55-46fe-bcf9-4d1e60d69c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.321831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-33a59742-cf55-46fe-bcf9-4d1e60d69c78 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:59.323008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-33a59742-cf55-46fe-bcf9-4d1e60d69c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:59.331554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1085/2137] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:59.335725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4ad7c-eb1b-48fb-8d06-3ca5a40c26fe None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:59.345781 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4ad7c-eb1b-48fb-8d06-3ca5a40c26fe None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:59.350280 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:59.350866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.351143 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.351412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.352053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:59.352354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:59.354063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:59.365244 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1053/2138] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:59.369394 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a59742-cf55-46fe-bcf9-4d1e60d69c78 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:59.379764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a59742-cf55-46fe-bcf9-4d1e60d69c78 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:59.384769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159e7d5-9409-437b-9e68-02953740e3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:59.385411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e7d5-9409-437b-9e68-02953740e3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.385697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e7d5-9409-437b-9e68-02953740e3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.386043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159e7d5-9409-437b-9e68-02953740e3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.386761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159e7d5-9409-437b-9e68-02953740e3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:59.389387 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159e7d5-9409-437b-9e68-02953740e3e2 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:59.462729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1086/2139] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:59 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:59.470340 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:59.480327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0155a39-d6fa-4bf4-8ebc-d976f77c209d admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:59.501619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:59.538527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac1bd3-8ac4-48c0-84c5-d57f1a723c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:59.539127 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac1bd3-8ac4-48c0-84c5-d57f1a723c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.539417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac1bd3-8ac4-48c0-84c5-d57f1a723c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.539722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac1bd3-8ac4-48c0-84c5-d57f1a723c32 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.540962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac1bd3-8ac4-48c0-84c5-d57f1a723c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:59.545974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac1bd3-8ac4-48c0-84c5-d57f1a723c32 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:59.656426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1054/2140] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:59 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:59.660640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159e7d5-9409-437b-9e68-02953740e3e2 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:59.671356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159e7d5-9409-437b-9e68-02953740e3e2 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:59.676030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdfed8-a06a-4062-a0d7-e73ea6be32d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:59.676628 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdfed8-a06a-4062-a0d7-e73ea6be32d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.676910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdfed8-a06a-4062-a0d7-e73ea6be32d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.677201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdfed8-a06a-4062-a0d7-e73ea6be32d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.677860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdfed8-a06a-4062-a0d7-e73ea6be32d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:59.682498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdfed8-a06a-4062-a0d7-e73ea6be32d6 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:59.689880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1087/2141] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:59.694088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac1bd3-8ac4-48c0-84c5-d57f1a723c32 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:59.704967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac1bd3-8ac4-48c0-84c5-d57f1a723c32 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:59.711107 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709fa2d-c1cb-4880-be71-3ce6ad051483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:59.711750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709fa2d-c1cb-4880-be71-3ce6ad051483 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.712050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709fa2d-c1cb-4880-be71-3ce6ad051483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.712321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709fa2d-c1cb-4880-be71-3ce6ad051483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.712963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709fa2d-c1cb-4880-be71-3ce6ad051483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:59.714619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709fa2d-c1cb-4880-be71-3ce6ad051483 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:59.725871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1055/2142] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:59.730196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdfed8-a06a-4062-a0d7-e73ea6be32d6 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:59.740194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdfed8-a06a-4062-a0d7-e73ea6be32d6 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:59.744751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c23b4d-0c47-448b-b079-356834de9670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:59.745418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c23b4d-0c47-448b-b079-356834de9670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.745693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c23b4d-0c47-448b-b079-356834de9670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.746006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c23b4d-0c47-448b-b079-356834de9670 admin admin] PATH_INFO: `/v3/projects/c3e40c73bfc4450c962974f03a542afd/users/da8ac5b4f2bf431995d5204ee6f9acd5/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.763022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c23b4d-0c47-448b-b079-356834de9670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e40c73bfc4450c962974f03a542afd, user_id=da8ac5b4f2bf431995d5204ee6f9acd5, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:59.766559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c23b4d-0c47-448b-b079-356834de9670 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:59.849748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1088/2143] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:59 2022] PUT /identity/v3/projects/c3e40c73bfc4450c962974f03a542afd/users/da8ac5b4f2bf431995d5204ee6f9acd5/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:59.870199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5a921-d418-4373-9d02-fab22b7826d2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:59.870493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5a921-d418-4373-9d02-fab22b7826d2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:59.870768 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5a921-d418-4373-9d02-fab22b7826d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:59.904364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d8a5a921-d418-4373-9d02-fab22b7826d2 None None] MFA Rules not processed for user `da8ac5b4f2bf431995d5204ee6f9acd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:59.905606 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d8a5a921-d418-4373-9d02-fab22b7826d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:59.979219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1056/2144] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:43:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:59.983788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c23b4d-0c47-448b-b079-356834de9670 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:00.042277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:00.060463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c23b4d-0c47-448b-b079-356834de9670 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:00.070971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a1ee6-7960-401d-be7a-ece633cb37ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:00.071698 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1ee6-7960-401d-be7a-ece633cb37ab admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:00.072192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1ee6-7960-401d-be7a-ece633cb37ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:00.072648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1ee6-7960-401d-be7a-ece633cb37ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:00.073372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a1ee6-7960-401d-be7a-ece633cb37ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:00.075655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a1ee6-7960-401d-be7a-ece633cb37ab admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:00.183199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1089/2145] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:59 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:44:00.187123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5a921-d418-4373-9d02-fab22b7826d2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:00.211040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5a921-d418-4373-9d02-fab22b7826d2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:00.277960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecd3c8-19b5-43f4-afcc-8853991dd61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:00.278867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd3c8-19b5-43f4-afcc-8853991dd61a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:00.279183 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd3c8-19b5-43f4-afcc-8853991dd61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:00.279442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd3c8-19b5-43f4-afcc-8853991dd61a admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:00.280615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecd3c8-19b5-43f4-afcc-8853991dd61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:00.282540 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecd3c8-19b5-43f4-afcc-8853991dd61a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:00.376354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1057/2146] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:44:00 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:44:00.380351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a1ee6-7960-401d-be7a-ece633cb37ab admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:00.391400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a1ee6-7960-401d-be7a-ece633cb37ab admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:00.395948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf7cbb-b5b9-4048-b88e-96d5f0f5e92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:00.396545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf7cbb-b5b9-4048-b88e-96d5f0f5e92a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:00.396824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf7cbb-b5b9-4048-b88e-96d5f0f5e92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:00.397094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf7cbb-b5b9-4048-b88e-96d5f0f5e92a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:00.397730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf7cbb-b5b9-4048-b88e-96d5f0f5e92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:00.399360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf7cbb-b5b9-4048-b88e-96d5f0f5e92a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:00.410654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1090/2147] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:00.414476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecd3c8-19b5-43f4-afcc-8853991dd61a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:00.424558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecd3c8-19b5-43f4-afcc-8853991dd61a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:00.430112 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6ab90-caeb-4427-9cd0-5520ca17fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:00.430772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6ab90-caeb-4427-9cd0-5520ca17fbd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:00.431056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6ab90-caeb-4427-9cd0-5520ca17fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:00.431327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6ab90-caeb-4427-9cd0-5520ca17fbd6 admin admin] PATH_INFO: `/v3/projects/d6be5d1429dc4b40a5a04911f98aa560/users/d7b60c949c0e4cbabdc6e9664049e75a/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:00.446898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6ab90-caeb-4427-9cd0-5520ca17fbd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6be5d1429dc4b40a5a04911f98aa560, user_id=d7b60c949c0e4cbabdc6e9664049e75a, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:00.449527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6ab90-caeb-4427-9cd0-5520ca17fbd6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:00.542263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:00.547885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1058/2148] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:44:00 2022] PUT /identity/v3/projects/d6be5d1429dc4b40a5a04911f98aa560/users/d7b60c949c0e4cbabdc6e9664049e75a/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:00.551824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf7cbb-b5b9-4048-b88e-96d5f0f5e92a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:00.562574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf7cbb-b5b9-4048-b88e-96d5f0f5e92a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:00.622032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2244e-4e05-4400-ac74-dd640b856d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:00.622633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2244e-4e05-4400-ac74-dd640b856d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:00.623154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2244e-4e05-4400-ac74-dd640b856d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:00.623918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2244e-4e05-4400-ac74-dd640b856d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:00.624556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2244e-4e05-4400-ac74-dd640b856d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:00.626145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2244e-4e05-4400-ac74-dd640b856d60 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:00.635650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1091/2149] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:00.640399 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6ab90-caeb-4427-9cd0-5520ca17fbd6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:00.651153 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6ab90-caeb-4427-9cd0-5520ca17fbd6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:00.655921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:00.656609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:00.656889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:00.657165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] PATH_INFO: `/v3/projects/d6be5d1429dc4b40a5a04911f98aa560/users/d7b60c949c0e4cbabdc6e9664049e75a/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:00.659178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6be5d1429dc4b40a5a04911f98aa560, user_id=d7b60c949c0e4cbabdc6e9664049e75a, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:00.662000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:00.672843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1059/2150] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:44:00 2022] PUT /identity/v3/projects/d6be5d1429dc4b40a5a04911f98aa560/users/d7b60c949c0e4cbabdc6e9664049e75a/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:00.683440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df70c-87a2-4546-b732-47a1f7a2789e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:00.683787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df70c-87a2-4546-b732-47a1f7a2789e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:00.684125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df70c-87a2-4546-b732-47a1f7a2789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:00.716425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d08df70c-87a2-4546-b732-47a1f7a2789e None None] MFA Rules not processed for user `d7b60c949c0e4cbabdc6e9664049e75a`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:00.717693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d08df70c-87a2-4546-b732-47a1f7a2789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:00.792121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1092/2151] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:44:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:00.802819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:00.804526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:00.863616 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c245be-750e-4ee5-9677-d182a228f742 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:00.867281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:00.867886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:00.868164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:00.868435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:00.878603 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:00.880064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:00.889505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1060/2152] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:01.146602 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08df70c-87a2-4546-b732-47a1f7a2789e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:01.158223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08df70c-87a2-4546-b732-47a1f7a2789e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:01.162831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb097f0-527e-4b63-b8fc-225a3df26d90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:01.163427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb097f0-527e-4b63-b8fc-225a3df26d90 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:01.163708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb097f0-527e-4b63-b8fc-225a3df26d90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:01.164029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb097f0-527e-4b63-b8fc-225a3df26d90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:01.211301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:01.236950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb097f0-527e-4b63-b8fc-225a3df26d90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:01.239400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb097f0-527e-4b63-b8fc-225a3df26d90 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:01.251421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1093/2153] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:02.907024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:02.909001 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:02.990582 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c69ce2-6d61-4707-8375-00f8b7645251 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:03.002193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:03.002957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:03.003636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:03.003636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:03.005638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:03.019432 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:03.021327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:03.034273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1061/2154] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:09.231622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db6675-cab8-43b0-b2e1-4463ac3c5fcf None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:09.232111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db6675-cab8-43b0-b2e1-4463ac3c5fcf None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:09.232201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db6675-cab8-43b0-b2e1-4463ac3c5fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:09.252015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-78db6675-cab8-43b0-b2e1-4463ac3c5fcf None None] MFA Rules not processed for user `205b19ccb80a45fd8ed36abd2c4c32c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:09.253205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-78db6675-cab8-43b0-b2e1-4463ac3c5fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:09.314190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1094/2155] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:44:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:09.929457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:09.931026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:09.997710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51685-1ab7-43e2-8bb6-b95aee649de3 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:10.002842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f3462-6f25-4964-a027-d4dad6316118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:10.003726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3462-6f25-4964-a027-d4dad6316118 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:10.004033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3462-6f25-4964-a027-d4dad6316118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:10.004393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3462-6f25-4964-a027-d4dad6316118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:10.006484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-855f3462-6f25-4964-a027-d4dad6316118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:10.017562 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f3462-6f25-4964-a027-d4dad6316118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:10.019021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f3462-6f25-4964-a027-d4dad6316118 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:10.029249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1062/2156] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:10.624242 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4782a-1078-4122-8946-94bceba63f98 None None] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:10.624806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4782a-1078-4122-8946-94bceba63f98 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:10.624806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4782a-1078-4122-8946-94bceba63f98 None None] PATH_INFO: `/` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:10.626949 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1095/2157] 10.222.0.44 () {54 vars in 921 bytes} [Wed Oct 12 12:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:44:10.633871 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33c26-9787-4832-b630-cb3873d55ff5 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:10.634627 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33c26-9787-4832-b630-cb3873d55ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:10.635101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33c26-9787-4832-b630-cb3873d55ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:10.667589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fac33c26-9787-4832-b630-cb3873d55ff5 None None] MFA Rules not processed for user `a0a7230c6da74e6484db3f260b4c58df`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:10.668810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fac33c26-9787-4832-b630-cb3873d55ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:10.679899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1063/2158] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Oct 12 12:44:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:10.703354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4782a-1078-4122-8946-94bceba63f98 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:10.742868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4782a-1078-4122-8946-94bceba63f98 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:10.759233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:10.759703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:10.759980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:10.760983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:10.782801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:10.789827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:10.808009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1096/2159] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:18.482397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ab02a-909d-4752-9ab4-797c0f59f66e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:18.482876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ab02a-909d-4752-9ab4-797c0f59f66e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:18.483072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ab02a-909d-4752-9ab4-797c0f59f66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:18.504415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-236ab02a-909d-4752-9ab4-797c0f59f66e None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:18.505843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-236ab02a-909d-4752-9ab4-797c0f59f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:18.512500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:18.576222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1064/2160] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:44:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:18.685034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:18.685034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:18.754277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ba1e-1b80-4a4c-a2b9-049bab451105 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:18.759326 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b66e5-53d9-4ec5-bdc5-735334730851 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:18.759928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b66e5-53d9-4ec5-bdc5-735334730851 service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:18.760203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b66e5-53d9-4ec5-bdc5-735334730851 service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:18.760474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b66e5-53d9-4ec5-bdc5-735334730851 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:18.769255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b66e5-53d9-4ec5-bdc5-735334730851 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:18.770850 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b66e5-53d9-4ec5-bdc5-735334730851 service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:18.778987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1097/2161] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:44:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:44:28.616021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=21 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:28.775622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=74 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:36.467423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ab02a-909d-4752-9ab4-797c0f59f66e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:36.468859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-236ab02a-909d-4752-9ab4-797c0f59f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:36.482537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ab02a-909d-4752-9ab4-797c0f59f66e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:36.484294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:36.484895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:36.485174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:36.485448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:36.487801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:36.550923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:36.552320 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:36.562375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1065/2162] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:46.190466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56841924-21c5-4300-8654-134da0a854d9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:46.190927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56841924-21c5-4300-8654-134da0a854d9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:46.191040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56841924-21c5-4300-8654-134da0a854d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:46.209817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-56841924-21c5-4300-8654-134da0a854d9 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:46.211037 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-56841924-21c5-4300-8654-134da0a854d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:46.218030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1098/2163] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:44:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:46.230434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:46.242348 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e02e7-bbb0-4892-8551-1297aad69979 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:46.247381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235d66b-1677-4780-bf2d-97568040eafb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:46.247977 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d66b-1677-4780-bf2d-97568040eafb service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:46.248255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d66b-1677-4780-bf2d-97568040eafb service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:46.248525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d66b-1677-4780-bf2d-97568040eafb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:46.256967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235d66b-1677-4780-bf2d-97568040eafb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:46.258570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235d66b-1677-4780-bf2d-97568040eafb service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:46.266599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1066/2164] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:44:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:44:52.224320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56841924-21c5-4300-8654-134da0a854d9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:52.288320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56841924-21c5-4300-8654-134da0a854d9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:52.292920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:52.293557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:52.293842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:52.294145 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] PATH_INFO: `/v3/users/da8ac5b4f2bf431995d5204ee6f9acd5` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:52.295371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=da8ac5b4f2bf431995d5204ee6f9acd5)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:52.297348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:52.447890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da8ac5b4f2bf431995d5204ee6f9acd5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:52.460624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da8ac5b4f2bf431995d5204ee6f9acd5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:52.461322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da8ac5b4f2bf431995d5204ee6f9acd5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:52.513964 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da8ac5b4f2bf431995d5204ee6f9acd5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:52.522423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da8ac5b4f2bf431995d5204ee6f9acd5'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:52.528438 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1099/2165] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:44:52 2022] DELETE /identity/v3/users/da8ac5b4f2bf431995d5204ee6f9acd5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:52.533106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd8c9-ace6-4b14-9afb-9db5eb2203bc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:52.533622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd8c9-ace6-4b14-9afb-9db5eb2203bc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:52.533732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd8c9-ace6-4b14-9afb-9db5eb2203bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:52.553407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-85abd8c9-ace6-4b14-9afb-9db5eb2203bc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:52.554628 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-85abd8c9-ace6-4b14-9afb-9db5eb2203bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:52.629104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1067/2166] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:44:52.639842 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:52.641365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:52.741390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eb29d-6161-464f-b8bd-6a5c74380d6e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:52.745114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50279186-2bce-4c87-b5e4-3714b99c6b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:52.745741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50279186-2bce-4c87-b5e4-3714b99c6b43 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:52.746166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50279186-2bce-4c87-b5e4-3714b99c6b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:52.746462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50279186-2bce-4c87-b5e4-3714b99c6b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:52.758540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50279186-2bce-4c87-b5e4-3714b99c6b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:52.760300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50279186-2bce-4c87-b5e4-3714b99c6b43 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:52.773395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1100/2167] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:53.367664 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abd8c9-ace6-4b14-9afb-9db5eb2203bc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:53.369240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-85abd8c9-ace6-4b14-9afb-9db5eb2203bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:53.384728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abd8c9-ace6-4b14-9afb-9db5eb2203bc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:53.389752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:53.390440 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:53.390719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:53.391021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] PATH_INFO: `/v3/projects/c3e40c73bfc4450c962974f03a542afd` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:53.392169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e40c73bfc4450c962974f03a542afd)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:53.394124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:53.484840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e40c73bfc4450c962974f03a542afd', 'request_id': 'req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:53.518708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e40c73bfc4450c962974f03a542afd', 'request_id': 'req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:53.518708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e40c73bfc4450c962974f03a542afd', 'request_id': 'req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:53.755804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] The token cache is being invalidate because project c3e40c73bfc4450c962974f03a542afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:44:53.756480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:53.756821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:53.762585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1068/2168] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:44:53 2022] DELETE /identity/v3/projects/c3e40c73bfc4450c962974f03a542afd => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:53.768117 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50279186-2bce-4c87-b5e4-3714b99c6b43 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:53.774044 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50279186-2bce-4c87-b5e4-3714b99c6b43 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:53.786151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:53.841989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:53.841989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:53.841989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:53.841989 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] PATH_INFO: `/v3/users/d7b60c949c0e4cbabdc6e9664049e75a` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:53.842648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7b60c949c0e4cbabdc6e9664049e75a)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:53.842648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:54.044456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7b60c949c0e4cbabdc6e9664049e75a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.079519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7b60c949c0e4cbabdc6e9664049e75a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.080486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7b60c949c0e4cbabdc6e9664049e75a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.142428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7b60c949c0e4cbabdc6e9664049e75a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.152283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6cf917dd-ae20-432a-bef2-9161614f7cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7b60c949c0e4cbabdc6e9664049e75a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.159928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1101/2169] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:44:53 2022] DELETE /identity/v3/users/d7b60c949c0e4cbabdc6e9664049e75a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:54.819747 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:54.822155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:54.832428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:54.836822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45b7f9-ddc6-4128-87c9-ff9d6cb4ff3b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:54.926722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:54.927398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:54.927745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:54.928103 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] PATH_INFO: `/v3/projects/d6be5d1429dc4b40a5a04911f98aa560` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:54.929315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6be5d1429dc4b40a5a04911f98aa560)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:54.931537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:55.201642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6be5d1429dc4b40a5a04911f98aa560', 'request_id': 'req-1c2f58af-9fdf-4971-9108-ea82b2e3e226'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.201642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6be5d1429dc4b40a5a04911f98aa560', 'request_id': 'req-1c2f58af-9fdf-4971-9108-ea82b2e3e226'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.201642 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6be5d1429dc4b40a5a04911f98aa560', 'request_id': 'req-1c2f58af-9fdf-4971-9108-ea82b2e3e226'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.376957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] The token cache is being invalidate because project d6be5d1429dc4b40a5a04911f98aa560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:44:55.377245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.377873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1c2f58af-9fdf-4971-9108-ea82b2e3e226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.379912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1069/2170] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:44:54 2022] DELETE /identity/v3/projects/d6be5d1429dc4b40a5a04911f98aa560 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:55.384874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a363d8-ebd5-467f-96f0-1e8a6f527c30 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.385411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a363d8-ebd5-467f-96f0-1e8a6f527c30 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.385518 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a363d8-ebd5-467f-96f0-1e8a6f527c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.407564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-69a363d8-ebd5-467f-96f0-1e8a6f527c30 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:55.408774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-69a363d8-ebd5-467f-96f0-1e8a6f527c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:55.471448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1102/2171] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:44:55.481978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da9a96-042e-46ff-a52b-98897a464583 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.481978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da9a96-042e-46ff-a52b-98897a464583 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.481978 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da9a96-042e-46ff-a52b-98897a464583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.503057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-21da9a96-042e-46ff-a52b-98897a464583 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:55.505160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-21da9a96-042e-46ff-a52b-98897a464583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:55.514071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1070/2172] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:55.518355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a363d8-ebd5-467f-96f0-1e8a6f527c30 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:55.529406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a363d8-ebd5-467f-96f0-1e8a6f527c30 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:55.534078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:55.534708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.534993 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.535302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.536002 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:44:55.536307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:55.537914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:55.552718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1103/2173] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:44:55.555078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da9a96-042e-46ff-a52b-98897a464583 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:55.564564 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da9a96-042e-46ff-a52b-98897a464583 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:55.570167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6ca9-79c7-4d51-9629-49fc06a018e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:55.570799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6ca9-79c7-4d51-9629-49fc06a018e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.571079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6ca9-79c7-4d51-9629-49fc06a018e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.571356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f6ca9-79c7-4d51-9629-49fc06a018e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.572018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f6ca9-79c7-4d51-9629-49fc06a018e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:55.573997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f6ca9-79c7-4d51-9629-49fc06a018e2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:55.670326 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1071/2174] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:44:55 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:44:55.676830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:55.687768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d4fe8-51c3-4269-95de-a68cc69276f7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:55.711330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:55.750856 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6141948-87f0-4db0-b9e9-c7c9051232e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:55.751419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141948-87f0-4db0-b9e9-c7c9051232e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.751699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141948-87f0-4db0-b9e9-c7c9051232e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.751972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141948-87f0-4db0-b9e9-c7c9051232e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.752622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6141948-87f0-4db0-b9e9-c7c9051232e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:55.755259 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6141948-87f0-4db0-b9e9-c7c9051232e9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:55.840952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1104/2175] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:44:55 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:44:55.845014 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6ca9-79c7-4d51-9629-49fc06a018e2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:55.856305 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f6ca9-79c7-4d51-9629-49fc06a018e2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:55.860863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10246c41-75be-4620-8c5f-b5b6d4f8efe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:55.861451 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10246c41-75be-4620-8c5f-b5b6d4f8efe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.861769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10246c41-75be-4620-8c5f-b5b6d4f8efe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.862221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10246c41-75be-4620-8c5f-b5b6d4f8efe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.862858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10246c41-75be-4620-8c5f-b5b6d4f8efe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:55.865077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10246c41-75be-4620-8c5f-b5b6d4f8efe0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:55.875316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1072/2176] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:55.879286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6141948-87f0-4db0-b9e9-c7c9051232e9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:55.889341 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6141948-87f0-4db0-b9e9-c7c9051232e9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:55.894718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55aa6a6-e0ce-4423-bdf5-df34ede043c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:55.895406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa6a6-e0ce-4423-bdf5-df34ede043c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.895688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa6a6-e0ce-4423-bdf5-df34ede043c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.896060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55aa6a6-e0ce-4423-bdf5-df34ede043c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.896899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55aa6a6-e0ce-4423-bdf5-df34ede043c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:55.898747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55aa6a6-e0ce-4423-bdf5-df34ede043c4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:55.913355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1105/2177] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:55.917480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10246c41-75be-4620-8c5f-b5b6d4f8efe0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:55.928558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10246c41-75be-4620-8c5f-b5b6d4f8efe0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:55.933368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a0ce5-2325-4240-98c7-d51c6fdc4e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:55.934138 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0ce5-2325-4240-98c7-d51c6fdc4e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.934488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0ce5-2325-4240-98c7-d51c6fdc4e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.934831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0ce5-2325-4240-98c7-d51c6fdc4e3c admin admin] PATH_INFO: `/v3/projects/46360d7e07de47c4b7c631b610ac6f3d/users/d406492653184e1298694907693c7459/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.951016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a0ce5-2325-4240-98c7-d51c6fdc4e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46360d7e07de47c4b7c631b610ac6f3d, user_id=d406492653184e1298694907693c7459, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:55.953909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a0ce5-2325-4240-98c7-d51c6fdc4e3c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:56.023544 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1073/2178] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:44:55 2022] PUT /identity/v3/projects/46360d7e07de47c4b7c631b610ac6f3d/users/d406492653184e1298694907693c7459/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:56.025892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9ea5-3ce4-4ad8-b236-31e1c5b6a82e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:56.026368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9ea5-3ce4-4ad8-b236-31e1c5b6a82e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:56.026651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e9ea5-3ce4-4ad8-b236-31e1c5b6a82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:56.059791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-217e9ea5-3ce4-4ad8-b236-31e1c5b6a82e None None] MFA Rules not processed for user `d406492653184e1298694907693c7459`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:56.061065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-217e9ea5-3ce4-4ad8-b236-31e1c5b6a82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:56.134954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1106/2179] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:44:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:56.139494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a0ce5-2325-4240-98c7-d51c6fdc4e3c admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:56.174244 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:56.208911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a0ce5-2325-4240-98c7-d51c6fdc4e3c admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:56.214531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2342d-c668-4dfc-9935-16bb564dc2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:56.215231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2342d-c668-4dfc-9935-16bb564dc2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:56.215583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2342d-c668-4dfc-9935-16bb564dc2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:56.215921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2342d-c668-4dfc-9935-16bb564dc2e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:56.216784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2342d-c668-4dfc-9935-16bb564dc2e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:56.218745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2342d-c668-4dfc-9935-16bb564dc2e9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:56.356792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1074/2180] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:44:56 2022] POST /identity/v3/projects => generated 369 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:44:56.360819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e9ea5-3ce4-4ad8-b236-31e1c5b6a82e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:56.371696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e9ea5-3ce4-4ad8-b236-31e1c5b6a82e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:56.439086 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a1c37-f5c7-4e2a-9217-0abad9cfede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:56.439851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a1c37-f5c7-4e2a-9217-0abad9cfede9 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:56.440323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a1c37-f5c7-4e2a-9217-0abad9cfede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:56.440635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a1c37-f5c7-4e2a-9217-0abad9cfede9 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:56.441484 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a1c37-f5c7-4e2a-9217-0abad9cfede9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:56.443724 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a1c37-f5c7-4e2a-9217-0abad9cfede9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:56.572241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1107/2181] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:44:56 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:44:56.576167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2342d-c668-4dfc-9935-16bb564dc2e9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:56.588473 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2342d-c668-4dfc-9935-16bb564dc2e9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:56.593438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a106f-89dd-4011-abad-f00a72a7618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:56.594233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a106f-89dd-4011-abad-f00a72a7618f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:56.594741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a106f-89dd-4011-abad-f00a72a7618f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:56.595114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a106f-89dd-4011-abad-f00a72a7618f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:56.595884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a106f-89dd-4011-abad-f00a72a7618f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:56.597794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a106f-89dd-4011-abad-f00a72a7618f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:56.608445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1075/2182] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:56.612662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a1c37-f5c7-4e2a-9217-0abad9cfede9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:56.622934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a1c37-f5c7-4e2a-9217-0abad9cfede9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:56.627938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a218d5d-14f5-4373-8018-d9253bc519f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:56.628799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a218d5d-14f5-4373-8018-d9253bc519f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:56.629561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a218d5d-14f5-4373-8018-d9253bc519f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:56.629831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a218d5d-14f5-4373-8018-d9253bc519f8 admin admin] PATH_INFO: `/v3/projects/466bda5ed68e48eb8117a695c423fe06/users/e9301839dbbb44f68d331dfacdb7cddc/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:56.645726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a218d5d-14f5-4373-8018-d9253bc519f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=466bda5ed68e48eb8117a695c423fe06, user_id=e9301839dbbb44f68d331dfacdb7cddc, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:56.650638 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a218d5d-14f5-4373-8018-d9253bc519f8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:56.651400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:56.731341 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1108/2183] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:44:56 2022] PUT /identity/v3/projects/466bda5ed68e48eb8117a695c423fe06/users/e9301839dbbb44f68d331dfacdb7cddc/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:44:56.735296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a106f-89dd-4011-abad-f00a72a7618f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:56.746066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a106f-89dd-4011-abad-f00a72a7618f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:56.805835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6def2d-10f7-47c4-a8fe-80a23d1536e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:56.806472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6def2d-10f7-47c4-a8fe-80a23d1536e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:56.806748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6def2d-10f7-47c4-a8fe-80a23d1536e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:56.807016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6def2d-10f7-47c4-a8fe-80a23d1536e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:56.807649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6def2d-10f7-47c4-a8fe-80a23d1536e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:56.809222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6def2d-10f7-47c4-a8fe-80a23d1536e4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:56.818831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1076/2184] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:56.823067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a218d5d-14f5-4373-8018-d9253bc519f8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:56.833653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a218d5d-14f5-4373-8018-d9253bc519f8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:56.839126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:56.840123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:56.840406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:56.840684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] PATH_INFO: `/v3/projects/466bda5ed68e48eb8117a695c423fe06/users/e9301839dbbb44f68d331dfacdb7cddc/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:56.842860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=466bda5ed68e48eb8117a695c423fe06, user_id=e9301839dbbb44f68d331dfacdb7cddc, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:56.845805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:56.857247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1109/2185] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:44:56 2022] PUT /identity/v3/projects/466bda5ed68e48eb8117a695c423fe06/users/e9301839dbbb44f68d331dfacdb7cddc/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:44:56.868325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a09e8-dd0f-4aa1-adc5-9fbc3c6644bd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:56.870095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a09e8-dd0f-4aa1-adc5-9fbc3c6644bd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:56.870095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a09e8-dd0f-4aa1-adc5-9fbc3c6644bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:56.896513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ff4a09e8-dd0f-4aa1-adc5-9fbc3c6644bd None None] MFA Rules not processed for user `e9301839dbbb44f68d331dfacdb7cddc`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:56.897702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ff4a09e8-dd0f-4aa1-adc5-9fbc3c6644bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:56.966148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1077/2186] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:44:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:44:56.978454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:56.980183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:57.039855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23290c6f-e6f0-4e11-859e-dcc85554f2ad admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:57.043738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b13bb-dacc-4e57-922e-180d95e98bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:57.044335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b13bb-dacc-4e57-922e-180d95e98bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:57.044613 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b13bb-dacc-4e57-922e-180d95e98bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:57.044885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b13bb-dacc-4e57-922e-180d95e98bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:57.053994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b13bb-dacc-4e57-922e-180d95e98bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:57.055409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b13bb-dacc-4e57-922e-180d95e98bd6 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:57.064880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1110/2187] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:59.727805 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a09e8-dd0f-4aa1-adc5-9fbc3c6644bd None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:59.759142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:59.792214 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a09e8-dd0f-4aa1-adc5-9fbc3c6644bd None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:59.796914 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:59.797544 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:59.797823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:59.798126 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] PATH_INFO: `/v3/users/d406492653184e1298694907693c7459` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:59.799443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d406492653184e1298694907693c7459)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:59.801354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:59.935073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd406492653184e1298694907693c7459'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:59.947539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd406492653184e1298694907693c7459'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.022624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd406492653184e1298694907693c7459'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.023249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd406492653184e1298694907693c7459'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.031226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd406492653184e1298694907693c7459'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.037858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1078/2188] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:44:59 2022] DELETE /identity/v3/users/d406492653184e1298694907693c7459 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:00.045115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc013e-c58b-48cf-8839-a33f8afacf29 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:00.045584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc013e-c58b-48cf-8839-a33f8afacf29 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:00.045703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc013e-c58b-48cf-8839-a33f8afacf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:00.065442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-56dc013e-c58b-48cf-8839-a33f8afacf29 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:00.070475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-56dc013e-c58b-48cf-8839-a33f8afacf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:00.129413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1111/2189] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:00.138877 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:00.140258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:00.197071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4caec-bff6-4d63-980d-8aa313e41595 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:00.200697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160380e9-4208-4719-a097-f878fdaafd05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:00.201289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160380e9-4208-4719-a097-f878fdaafd05 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:00.201568 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160380e9-4208-4719-a097-f878fdaafd05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:00.201842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160380e9-4208-4719-a097-f878fdaafd05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:00.211249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160380e9-4208-4719-a097-f878fdaafd05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:00.217268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160380e9-4208-4719-a097-f878fdaafd05 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:00.222251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1079/2190] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:00.573204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc013e-c58b-48cf-8839-a33f8afacf29 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:00.574655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-56dc013e-c58b-48cf-8839-a33f8afacf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:00.588413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc013e-c58b-48cf-8839-a33f8afacf29 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:00.593591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:00.594306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:00.594588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:00.594914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] PATH_INFO: `/v3/projects/46360d7e07de47c4b7c631b610ac6f3d` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:00.596077 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46360d7e07de47c4b7c631b610ac6f3d)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:00.598089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:00.682857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46360d7e07de47c4b7c631b610ac6f3d', 'request_id': 'req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.688739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46360d7e07de47c4b7c631b610ac6f3d', 'request_id': 'req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.696269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46360d7e07de47c4b7c631b610ac6f3d', 'request_id': 'req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.770275 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] The token cache is being invalidate because project 46360d7e07de47c4b7c631b610ac6f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:00.770588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.771212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:00.772721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1112/2191] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:00 2022] DELETE /identity/v3/projects/46360d7e07de47c4b7c631b610ac6f3d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:00.776658 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160380e9-4208-4719-a097-f878fdaafd05 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:00.786460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160380e9-4208-4719-a097-f878fdaafd05 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:00.831112 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:00.842622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:00.843250 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:00.843533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:00.843806 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] PATH_INFO: `/v3/users/e9301839dbbb44f68d331dfacdb7cddc` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:00.844980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9301839dbbb44f68d331dfacdb7cddc)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:00.846937 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:01.090119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9301839dbbb44f68d331dfacdb7cddc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.103739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9301839dbbb44f68d331dfacdb7cddc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.159972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9301839dbbb44f68d331dfacdb7cddc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.160661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9301839dbbb44f68d331dfacdb7cddc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.171503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cd7a14a2-9927-4ff3-a176-a48a20bbe355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9301839dbbb44f68d331dfacdb7cddc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.182040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1080/2192] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:00 2022] DELETE /identity/v3/users/e9301839dbbb44f68d331dfacdb7cddc => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:01.599225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:01.599756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:01.614826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74cbf0-130f-412d-92c4-e728d5c9ebc4 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:01.621277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:01.673457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:01.674116 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:01.674399 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:01.674673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] PATH_INFO: `/v3/projects/466bda5ed68e48eb8117a695c423fe06` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:01.675784 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=466bda5ed68e48eb8117a695c423fe06)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:01.678243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:01.765233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466bda5ed68e48eb8117a695c423fe06', 'request_id': 'req-c5595b7e-05d3-4550-96e9-2c942f0e0984'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.770871 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466bda5ed68e48eb8117a695c423fe06', 'request_id': 'req-c5595b7e-05d3-4550-96e9-2c942f0e0984'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.777382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466bda5ed68e48eb8117a695c423fe06', 'request_id': 'req-c5595b7e-05d3-4550-96e9-2c942f0e0984'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.822170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] The token cache is being invalidate because project 466bda5ed68e48eb8117a695c423fe06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:01.822388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.823139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c5595b7e-05d3-4550-96e9-2c942f0e0984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:01.824612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1113/2193] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:01 2022] DELETE /identity/v3/projects/466bda5ed68e48eb8117a695c423fe06 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:01.830372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a94d4-67ad-4457-aa0c-779e48832460 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:01.830839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a94d4-67ad-4457-aa0c-779e48832460 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:01.830935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a94d4-67ad-4457-aa0c-779e48832460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:01.851216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-855a94d4-67ad-4457-aa0c-779e48832460 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:01.854014 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-855a94d4-67ad-4457-aa0c-779e48832460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:01.909788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1081/2194] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:01.918105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fae52-e1f5-4e03-b884-228d582b223b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:01.918396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fae52-e1f5-4e03-b884-228d582b223b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:01.918655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fae52-e1f5-4e03-b884-228d582b223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:01.938559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5a4fae52-e1f5-4e03-b884-228d582b223b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:01.939742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5a4fae52-e1f5-4e03-b884-228d582b223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:01.947563 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1114/2195] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:01.952193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a94d4-67ad-4457-aa0c-779e48832460 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:01.962120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a94d4-67ad-4457-aa0c-779e48832460 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:01.966836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:01.967605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:01.967696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:01.967943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:01.968646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:01.968994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:01.970674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:01.983201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1082/2196] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:45:01.987485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fae52-e1f5-4e03-b884-228d582b223b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:01.997836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fae52-e1f5-4e03-b884-228d582b223b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:02.002778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1260c6d-9f6c-4898-b3d8-b3dc2d6e069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:02.003367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1260c6d-9f6c-4898-b3d8-b3dc2d6e069e admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.003647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1260c6d-9f6c-4898-b3d8-b3dc2d6e069e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.003918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1260c6d-9f6c-4898-b3d8-b3dc2d6e069e admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:02.004543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1260c6d-9f6c-4898-b3d8-b3dc2d6e069e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:02.006479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1260c6d-9f6c-4898-b3d8-b3dc2d6e069e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:02.085057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1115/2197] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:01 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:02.085221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:02.095078 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b8eec-e838-4310-9de7-aafc43abeee6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:02.144508 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:02.151245 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3d98b-53c6-4b44-9ca8-5403885add98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:02.153967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3d98b-53c6-4b44-9ca8-5403885add98 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.153967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3d98b-53c6-4b44-9ca8-5403885add98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.153967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3d98b-53c6-4b44-9ca8-5403885add98 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:02.153967 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3d98b-53c6-4b44-9ca8-5403885add98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:02.155198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3d98b-53c6-4b44-9ca8-5403885add98 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:02.222468 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1083/2198] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:02 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:02.227364 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1260c6d-9f6c-4898-b3d8-b3dc2d6e069e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:02.237686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1260c6d-9f6c-4898-b3d8-b3dc2d6e069e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:02.242336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebbdb5-1153-44ec-bf2b-20603d61020c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:02.242924 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebbdb5-1153-44ec-bf2b-20603d61020c admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.243289 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebbdb5-1153-44ec-bf2b-20603d61020c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.243564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edebbdb5-1153-44ec-bf2b-20603d61020c admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:02.244201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebbdb5-1153-44ec-bf2b-20603d61020c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:02.245777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edebbdb5-1153-44ec-bf2b-20603d61020c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:02.256422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1116/2199] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:02.260412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3d98b-53c6-4b44-9ca8-5403885add98 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:02.270414 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3d98b-53c6-4b44-9ca8-5403885add98 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:02.275584 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934e10f-c54a-4a55-acd8-0c5784e5c8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:02.276178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e10f-c54a-4a55-acd8-0c5784e5c8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.276840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e10f-c54a-4a55-acd8-0c5784e5c8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.276840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934e10f-c54a-4a55-acd8-0c5784e5c8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:02.277335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934e10f-c54a-4a55-acd8-0c5784e5c8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:02.278970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934e10f-c54a-4a55-acd8-0c5784e5c8ac admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:02.291210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1084/2200] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:02.294574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebbdb5-1153-44ec-bf2b-20603d61020c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:02.304603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edebbdb5-1153-44ec-bf2b-20603d61020c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:02.309333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:02.309996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.310273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.310543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] PATH_INFO: `/v3/projects/0d220ae728eb425cb86280611b43da16/users/fd3873b2529045a3b97b70e069a418a1/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:02.325295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d220ae728eb425cb86280611b43da16, user_id=fd3873b2529045a3b97b70e069a418a1, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:02.327970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:02.400809 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1117/2201] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:02 2022] PUT /identity/v3/projects/0d220ae728eb425cb86280611b43da16/users/fd3873b2529045a3b97b70e069a418a1/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:02.412959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849bd4a-8289-40d7-b2ca-44afc48c04f0 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.413240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849bd4a-8289-40d7-b2ca-44afc48c04f0 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.413513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849bd4a-8289-40d7-b2ca-44afc48c04f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:02.444591 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b849bd4a-8289-40d7-b2ca-44afc48c04f0 None None] MFA Rules not processed for user `fd3873b2529045a3b97b70e069a418a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:02.445803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b849bd4a-8289-40d7-b2ca-44afc48c04f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:02.508794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1085/2202] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:02.523049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:02.524972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:02.565882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:02.582512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad548a-f9c1-40cf-a8b4-32f6e801db61 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:02.586193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:02.586804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.587091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.587996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:02.596879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:02.598401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:02.607818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1118/2203] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:02.833155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda467e-5b01-4ea5-9003-f4a898ae387a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.833631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda467e-5b01-4ea5-9003-f4a898ae387a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.833742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda467e-5b01-4ea5-9003-f4a898ae387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:02.854053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8eda467e-5b01-4ea5-9003-f4a898ae387a None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:02.855257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8eda467e-5b01-4ea5-9003-f4a898ae387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:02.910167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1086/2204] 10.222.0.44 () {58 vars in 967 bytes} [Wed Oct 12 12:45:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:02.923021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:02.924395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:02.992593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5ac0c-5765-4170-9af3-f701499704bd service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:02.997628 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b023d-1e0b-4e9d-88d3-3b95ef93d0b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:02.998257 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b023d-1e0b-4e9d-88d3-3b95ef93d0b2 service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:02.998540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b023d-1e0b-4e9d-88d3-3b95ef93d0b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:02.998807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b023d-1e0b-4e9d-88d3-3b95ef93d0b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:03.007701 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b023d-1e0b-4e9d-88d3-3b95ef93d0b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:03.009260 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b023d-1e0b-4e9d-88d3-3b95ef93d0b2 service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:03.017396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1119/2205] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:45:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 12:45:12.903343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:13.014112 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=42 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:25.160210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda467e-5b01-4ea5-9003-f4a898ae387a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:25.161609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8eda467e-5b01-4ea5-9003-f4a898ae387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:25.217493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda467e-5b01-4ea5-9003-f4a898ae387a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:25.221090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6e079-4596-4709-9021-a989e14c49ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:25.221690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6e079-4596-4709-9021-a989e14c49ff service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:25.222001 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6e079-4596-4709-9021-a989e14c49ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:25.222276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6e079-4596-4709-9021-a989e14c49ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:25.223987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-40b6e079-4596-4709-9021-a989e14c49ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:25.286012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6e079-4596-4709-9021-a989e14c49ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:25.287496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b6e079-4596-4709-9021-a989e14c49ff service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:25.298003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1087/2206] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:27.520628 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b023d-1e0b-4e9d-88d3-3b95ef93d0b2 service swift] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:27.531611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b023d-1e0b-4e9d-88d3-3b95ef93d0b2 service swift] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:27.536726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:27.536915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:27.537070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:27.537365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] PATH_INFO: `/v3/users/fd3873b2529045a3b97b70e069a418a1` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:27.538502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3873b2529045a3b97b70e069a418a1)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:27.540427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:27.741402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3873b2529045a3b97b70e069a418a1'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:27.755165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3873b2529045a3b97b70e069a418a1'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:27.755912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3873b2529045a3b97b70e069a418a1'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:27.846592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3873b2529045a3b97b70e069a418a1'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:27.854931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3873b2529045a3b97b70e069a418a1'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:27.861225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1120/2207] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:27 2022] DELETE /identity/v3/users/fd3873b2529045a3b97b70e069a418a1 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:27.865634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bee7f-026f-43e9-b6f8-ce862b10d596 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:27.866172 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bee7f-026f-43e9-b6f8-ce862b10d596 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:27.866278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bee7f-026f-43e9-b6f8-ce862b10d596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:27.888873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-876bee7f-026f-43e9-b6f8-ce862b10d596 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:27.890136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-876bee7f-026f-43e9-b6f8-ce862b10d596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:27.957696 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1088/2208] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:27.974622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:27.974622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:28.037289 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3a526-adf5-4186-a47f-c20054d17515 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:28.041732 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87afcaf-6df9-493c-8f6f-c1b8b660006e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:28.044504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87afcaf-6df9-493c-8f6f-c1b8b660006e service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:28.044790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87afcaf-6df9-493c-8f6f-c1b8b660006e service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:28.045068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87afcaf-6df9-493c-8f6f-c1b8b660006e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:28.058618 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87afcaf-6df9-493c-8f6f-c1b8b660006e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:28.060243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87afcaf-6df9-493c-8f6f-c1b8b660006e service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:28.071631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1121/2209] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:45:28.574529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bee7f-026f-43e9-b6f8-ce862b10d596 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:28.576665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-876bee7f-026f-43e9-b6f8-ce862b10d596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:28.593513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bee7f-026f-43e9-b6f8-ce862b10d596 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:28.598388 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:28.600144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:28.600144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:28.600144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] PATH_INFO: `/v3/projects/0d220ae728eb425cb86280611b43da16` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:28.600738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d220ae728eb425cb86280611b43da16)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:28.603270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:28.729221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:28.745228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d220ae728eb425cb86280611b43da16', 'request_id': 'req-359d1d3d-9851-4978-8e46-006355accd31'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:28.752607 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d220ae728eb425cb86280611b43da16', 'request_id': 'req-359d1d3d-9851-4978-8e46-006355accd31'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:28.768709 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d220ae728eb425cb86280611b43da16', 'request_id': 'req-359d1d3d-9851-4978-8e46-006355accd31'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:28.846565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] The token cache is being invalidate because project 0d220ae728eb425cb86280611b43da16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:28.846859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:28.847510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:28.849026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1089/2210] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:28 2022] DELETE /identity/v3/projects/0d220ae728eb425cb86280611b43da16 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:28.854820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8afac-40ff-4242-a2d7-8f455402bd9e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:28.855303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8afac-40ff-4242-a2d7-8f455402bd9e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:28.855418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8afac-40ff-4242-a2d7-8f455402bd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:28.864351 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:28.866229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:28.891567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-e0a8afac-40ff-4242-a2d7-8f455402bd9e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:28.894141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e0a8afac-40ff-4242-a2d7-8f455402bd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:28.955735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d1d3d-9851-4978-8e46-006355accd31 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:28.962828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:28.963452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:28.963719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:28.964044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:28.965949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:28.978913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1122/2211] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:28.986292 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e9bbe-faaf-4c32-87f4-1f336befbe32 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:28.986582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e9bbe-faaf-4c32-87f4-1f336befbe32 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:28.987436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e9bbe-faaf-4c32-87f4-1f336befbe32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:28.997093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:28.999254 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.012819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1090/2212] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:29.013832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.020578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-434e9bbe-faaf-4c32-87f4-1f336befbe32 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:29.020578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-434e9bbe-faaf-4c32-87f4-1f336befbe32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:29.022312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:29.026012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1123/2213] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:29.030580 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e9bbe-faaf-4c32-87f4-1f336befbe32 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.041215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaad8bb-fbd8-406a-a7e3-0dc424f774c4 service placement] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.046227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e9bbe-faaf-4c32-87f4-1f336befbe32 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.048142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.049012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.049352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.049685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] PATH_INFO: `/v3/users/39543aedd1834f68a9d0a4ef174822c2` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.051054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39543aedd1834f68a9d0a4ef174822c2)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.054000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.054000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.054000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.054000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.054334 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:29.054427 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.054532 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.055949 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.059709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:29.071331 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1124/2214] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:45:29.074255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.084492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8eca52-7874-47d0-a1db-04131fbc1468 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.090892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baeae8-5eb7-41ce-a15d-f34e703c5237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.091492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baeae8-5eb7-41ce-a15d-f34e703c5237 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.091765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baeae8-5eb7-41ce-a15d-f34e703c5237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.092035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baeae8-5eb7-41ce-a15d-f34e703c5237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.092666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baeae8-5eb7-41ce-a15d-f34e703c5237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.094630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baeae8-5eb7-41ce-a15d-f34e703c5237 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.186681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1125/2215] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:29 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:45:29.190728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baeae8-5eb7-41ce-a15d-f34e703c5237 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.253876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baeae8-5eb7-41ce-a15d-f34e703c5237 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.258636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ac1ec-e896-4666-bfc5-952d10961f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.259229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ac1ec-e896-4666-bfc5-952d10961f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.259509 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ac1ec-e896-4666-bfc5-952d10961f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.259784 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ac1ec-e896-4666-bfc5-952d10961f53 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.260406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ac1ec-e896-4666-bfc5-952d10961f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.262408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ac1ec-e896-4666-bfc5-952d10961f53 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.281570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39543aedd1834f68a9d0a4ef174822c2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.295079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39543aedd1834f68a9d0a4ef174822c2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.349673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1126/2216] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:29 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:29.354585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ac1ec-e896-4666-bfc5-952d10961f53 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.364081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ac1ec-e896-4666-bfc5-952d10961f53 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.369011 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.369625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.369906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.370216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.370876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.372336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39543aedd1834f68a9d0a4ef174822c2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.372503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.372949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39543aedd1834f68a9d0a4ef174822c2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.381101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39543aedd1834f68a9d0a4ef174822c2'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.385868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1127/2217] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:45:29.386872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.387925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1091/2218] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:29 2022] DELETE /identity/v3/users/39543aedd1834f68a9d0a4ef174822c2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:29.388543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:29.402329 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b7870-ddf8-4c09-a76f-d4daedec1cec admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.464719 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28165704-d865-4566-bbcb-ec927ce172e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.465325 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165704-d865-4566-bbcb-ec927ce172e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.465603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165704-d865-4566-bbcb-ec927ce172e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.465875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165704-d865-4566-bbcb-ec927ce172e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.466550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28165704-d865-4566-bbcb-ec927ce172e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.468134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28165704-d865-4566-bbcb-ec927ce172e3 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.478800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1128/2219] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:29.483039 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.493116 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3674b-a772-40cc-8d70-7fbf2a074e14 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.497789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.498546 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.498828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.499104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] PATH_INFO: `/v3/projects/cb20ae49886b4a97a08c8dd5b60c088c/users/93e75a5e2554402c9f15d4db876a3560/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.514884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb20ae49886b4a97a08c8dd5b60c088c, user_id=93e75a5e2554402c9f15d4db876a3560, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.517822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.571983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1092/2220] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:29 2022] PUT /identity/v3/projects/cb20ae49886b4a97a08c8dd5b60c088c/users/93e75a5e2554402c9f15d4db876a3560/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:29.637576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfbb8f-a6af-4f95-8f96-52efd6d79334 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.637874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfbb8f-a6af-4f95-8f96-52efd6d79334 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.638175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfbb8f-a6af-4f95-8f96-52efd6d79334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.666691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-28cfbb8f-a6af-4f95-8f96-52efd6d79334 None None] MFA Rules not processed for user `93e75a5e2554402c9f15d4db876a3560`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:29.667976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-28cfbb8f-a6af-4f95-8f96-52efd6d79334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:29.670484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.672598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:29.690347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:29.712144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:29.737764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1129/2221] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:29.741952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfbb8f-a6af-4f95-8f96-52efd6d79334 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.746228 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d6f89-0ee1-4077-882f-09382d6c8918 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.751255 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.751880 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.752157 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.752490 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] PATH_INFO: `/v3/projects/fb8207ad9abf449b99382a2586d4588a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.752687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfbb8f-a6af-4f95-8f96-52efd6d79334 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.753951 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8207ad9abf449b99382a2586d4588a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.756422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.758803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f312729-04ac-4149-9ddd-3348034adcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.759403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f312729-04ac-4149-9ddd-3348034adcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.759704 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f312729-04ac-4149-9ddd-3348034adcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.759975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f312729-04ac-4149-9ddd-3348034adcb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.769683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f312729-04ac-4149-9ddd-3348034adcb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.769683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f312729-04ac-4149-9ddd-3348034adcb8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.853653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1130/2222] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:29 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:45:29.857487 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f312729-04ac-4149-9ddd-3348034adcb8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:29.894185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8207ad9abf449b99382a2586d4588a', 'request_id': 'req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.899576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8207ad9abf449b99382a2586d4588a', 'request_id': 'req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.905866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8207ad9abf449b99382a2586d4588a', 'request_id': 'req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.919826 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f312729-04ac-4149-9ddd-3348034adcb8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:29.977038 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:29.977647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.977926 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.978256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:29.978886 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:29.980780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:29.989040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] The token cache is being invalidate because project fb8207ad9abf449b99382a2586d4588a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:29.989334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.990074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bcd3745c-0abd-4a38-9d33-1c8856f5f1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:29.991723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1093/2223] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:29 2022] DELETE /identity/v3/projects/fb8207ad9abf449b99382a2586d4588a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:29.997251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0e6d-e79a-4431-82ee-6fa0a948eaab None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:29.997838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0e6d-e79a-4431-82ee-6fa0a948eaab None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:29.998134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0e6d-e79a-4431-82ee-6fa0a948eaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.019493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6abf0e6d-e79a-4431-82ee-6fa0a948eaab None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:30.020778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6abf0e6d-e79a-4431-82ee-6fa0a948eaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:30.028894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1094/2224] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:30.037464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540bee9-55ad-4585-864a-969d5831fc84 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.037742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540bee9-55ad-4585-864a-969d5831fc84 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.038046 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540bee9-55ad-4585-864a-969d5831fc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.057659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5540bee9-55ad-4585-864a-969d5831fc84 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:30.058868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5540bee9-55ad-4585-864a-969d5831fc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:30.066708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1095/2225] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:30.071632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540bee9-55ad-4585-864a-969d5831fc84 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.080995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540bee9-55ad-4585-864a-969d5831fc84 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.085750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.086374 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.086652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.086921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.090183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1131/2226] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:29 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:45:30.090333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:30.090333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.090333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.091258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.092834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:30.103959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1096/2227] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:45:30.109373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.113853 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebdd60-db61-420e-9be7-c61a41f7d4eb admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.119125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac64f5-38bc-4228-8884-73da26300d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.119730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac64f5-38bc-4228-8884-73da26300d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.120007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac64f5-38bc-4228-8884-73da26300d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.120273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac64f5-38bc-4228-8884-73da26300d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.120910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac64f5-38bc-4228-8884-73da26300d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.122565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac64f5-38bc-4228-8884-73da26300d0e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.122688 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2596fb-74ec-4d7f-a30e-399b7d739aff admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.128156 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98e175-2668-4748-bfe8-ec6b89203742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.128788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98e175-2668-4748-bfe8-ec6b89203742 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.129426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98e175-2668-4748-bfe8-ec6b89203742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.129426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98e175-2668-4748-bfe8-ec6b89203742 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.129994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98e175-2668-4748-bfe8-ec6b89203742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.131886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98e175-2668-4748-bfe8-ec6b89203742 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.133916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1132/2228] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:30.138083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac64f5-38bc-4228-8884-73da26300d0e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.148076 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac64f5-38bc-4228-8884-73da26300d0e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.152713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145977a-174b-47bf-af46-83d507e1d4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.153344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145977a-174b-47bf-af46-83d507e1d4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.153624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145977a-174b-47bf-af46-83d507e1d4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.153895 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145977a-174b-47bf-af46-83d507e1d4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.154558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145977a-174b-47bf-af46-83d507e1d4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.156144 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145977a-174b-47bf-af46-83d507e1d4e1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.166837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1133/2229] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:45:30.170948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145977a-174b-47bf-af46-83d507e1d4e1 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.180803 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145977a-174b-47bf-af46-83d507e1d4e1 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.185466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45594f57-06c0-43a3-9764-b7aa9d851036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.186102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594f57-06c0-43a3-9764-b7aa9d851036 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.186383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594f57-06c0-43a3-9764-b7aa9d851036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.186658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594f57-06c0-43a3-9764-b7aa9d851036 admin admin] PATH_INFO: `/v3/projects/0d9cf0a58e0d4667b794ced6ef704cab/users/120c62404f544d09a43e36078f01bd3b/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.202711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594f57-06c0-43a3-9764-b7aa9d851036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d9cf0a58e0d4667b794ced6ef704cab, user_id=120c62404f544d09a43e36078f01bd3b, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.205347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45594f57-06c0-43a3-9764-b7aa9d851036 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.235855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1097/2230] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:30 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:30.239788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98e175-2668-4748-bfe8-ec6b89203742 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.291360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:30.301484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98e175-2668-4748-bfe8-ec6b89203742 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.304703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1134/2231] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:30 2022] PUT /identity/v3/projects/0d9cf0a58e0d4667b794ced6ef704cab/users/120c62404f544d09a43e36078f01bd3b/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:30.315564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed489f-2576-4fb1-bf16-e49293d82f21 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.317152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed489f-2576-4fb1-bf16-e49293d82f21 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.317152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed489f-2576-4fb1-bf16-e49293d82f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.329126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:30.343507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-77ed489f-2576-4fb1-bf16-e49293d82f21 None None] MFA Rules not processed for user `120c62404f544d09a43e36078f01bd3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:30.344720 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-77ed489f-2576-4fb1-bf16-e49293d82f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:30.358990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede539-5c2b-43e9-bda5-5d62c665e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.359599 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede539-5c2b-43e9-bda5-5d62c665e7ee admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.359883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede539-5c2b-43e9-bda5-5d62c665e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.360158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede539-5c2b-43e9-bda5-5d62c665e7ee admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.360803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede539-5c2b-43e9-bda5-5d62c665e7ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.362814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede539-5c2b-43e9-bda5-5d62c665e7ee admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.411224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1135/2232] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:30.416194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed489f-2576-4fb1-bf16-e49293d82f21 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.426162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed489f-2576-4fb1-bf16-e49293d82f21 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.431337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7619f9-097b-448c-bdea-0443ea271b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.431911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7619f9-097b-448c-bdea-0443ea271b9c admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.432194 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7619f9-097b-448c-bdea-0443ea271b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.432469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7619f9-097b-448c-bdea-0443ea271b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.433089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7619f9-097b-448c-bdea-0443ea271b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.435055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7619f9-097b-448c-bdea-0443ea271b9c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.480860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1098/2233] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:30 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:30.484901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede539-5c2b-43e9-bda5-5d62c665e7ee admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.495610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede539-5c2b-43e9-bda5-5d62c665e7ee admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.501077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6d2b4-bb01-479b-9b1b-0df1d0791625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.501617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6d2b4-bb01-479b-9b1b-0df1d0791625 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.501899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6d2b4-bb01-479b-9b1b-0df1d0791625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.502213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6d2b4-bb01-479b-9b1b-0df1d0791625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.502874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6d2b4-bb01-479b-9b1b-0df1d0791625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.504456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6d2b4-bb01-479b-9b1b-0df1d0791625 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.514323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1099/2234] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:30.517927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6d2b4-bb01-479b-9b1b-0df1d0791625 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.527064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6d2b4-bb01-479b-9b1b-0df1d0791625 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.531625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2ef25-36fd-4226-8952-302b240adadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.532158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ef25-36fd-4226-8952-302b240adadf admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.532437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ef25-36fd-4226-8952-302b240adadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.532710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2ef25-36fd-4226-8952-302b240adadf admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.533307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2ef25-36fd-4226-8952-302b240adadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.534828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2ef25-36fd-4226-8952-302b240adadf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.545303 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1100/2235] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:45:30.548834 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2ef25-36fd-4226-8952-302b240adadf admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.558580 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2ef25-36fd-4226-8952-302b240adadf admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.563352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2c043-8ce0-43cb-9e97-d4db0a3ae282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.563959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2c043-8ce0-43cb-9e97-d4db0a3ae282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.564239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2c043-8ce0-43cb-9e97-d4db0a3ae282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.564510 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2c043-8ce0-43cb-9e97-d4db0a3ae282 admin admin] PATH_INFO: `/v3/projects/7f1531c48a844620bdbb4c6de446071e/users/3f701505e4d24954a01dc8252944320c/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.568104 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1136/2236] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:30 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:45:30.575435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7619f9-097b-448c-bdea-0443ea271b9c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.579922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2c043-8ce0-43cb-9e97-d4db0a3ae282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1531c48a844620bdbb4c6de446071e, user_id=3f701505e4d24954a01dc8252944320c, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.582618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2c043-8ce0-43cb-9e97-d4db0a3ae282 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.637114 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7619f9-097b-448c-bdea-0443ea271b9c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.641857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf1c6f-4515-4d91-ae0b-d65b26bfbfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.642481 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf1c6f-4515-4d91-ae0b-d65b26bfbfcd admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.642762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf1c6f-4515-4d91-ae0b-d65b26bfbfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.643035 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf1c6f-4515-4d91-ae0b-d65b26bfbfcd admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.643663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf1c6f-4515-4d91-ae0b-d65b26bfbfcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.645605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf1c6f-4515-4d91-ae0b-d65b26bfbfcd admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.654786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1101/2237] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:30 2022] PUT /identity/v3/projects/7f1531c48a844620bdbb4c6de446071e/users/3f701505e4d24954a01dc8252944320c/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:30.666519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a679-79d5-4cdd-bb8e-cd933c6d817c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.666806 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a679-79d5-4cdd-bb8e-cd933c6d817c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.667084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0a679-79d5-4cdd-bb8e-cd933c6d817c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.694291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-90e0a679-79d5-4cdd-bb8e-cd933c6d817c None None] MFA Rules not processed for user `3f701505e4d24954a01dc8252944320c`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:30.695524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-90e0a679-79d5-4cdd-bb8e-cd933c6d817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:30.729472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1137/2238] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:30 2022] POST /identity/v3/users => generated 298 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:30.730689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf1c6f-4515-4d91-ae0b-d65b26bfbfcd admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.740630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf1c6f-4515-4d91-ae0b-d65b26bfbfcd admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.762171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1102/2239] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:30.774183 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0a679-79d5-4cdd-bb8e-cd933c6d817c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.775762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-90e0a679-79d5-4cdd-bb8e-cd933c6d817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:30.799910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4facf2-d2b0-4e89-b070-f8d1ebbd9732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.800523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4facf2-d2b0-4e89-b070-f8d1ebbd9732 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.800815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4facf2-d2b0-4e89-b070-f8d1ebbd9732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.801089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4facf2-d2b0-4e89-b070-f8d1ebbd9732 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.801767 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4facf2-d2b0-4e89-b070-f8d1ebbd9732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.803457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4facf2-d2b0-4e89-b070-f8d1ebbd9732 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.809482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:30.813766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:30.814066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1138/2240] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:30.818007 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4facf2-d2b0-4e89-b070-f8d1ebbd9732 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.827683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4facf2-d2b0-4e89-b070-f8d1ebbd9732 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.832479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025aa64c-c0ac-4e77-812e-e2db2b31af36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.834693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025aa64c-c0ac-4e77-812e-e2db2b31af36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.834693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025aa64c-c0ac-4e77-812e-e2db2b31af36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.834693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025aa64c-c0ac-4e77-812e-e2db2b31af36 admin admin] PATH_INFO: `/v3/projects/fe9cc4682d1a4df8ad1a20f09080f016/users/390a29730a7a401798a8657fca1f0fe9/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.849108 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025aa64c-c0ac-4e77-812e-e2db2b31af36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9cc4682d1a4df8ad1a20f09080f016, user_id=390a29730a7a401798a8657fca1f0fe9, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.852378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025aa64c-c0ac-4e77-812e-e2db2b31af36 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.856392 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0a679-79d5-4cdd-bb8e-cd933c6d817c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.862189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c5927-97fa-4490-b685-ba7bb1718bee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.863352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c5927-97fa-4490-b685-ba7bb1718bee service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.863352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c5927-97fa-4490-b685-ba7bb1718bee service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.863352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c5927-97fa-4490-b685-ba7bb1718bee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:30.871128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c5927-97fa-4490-b685-ba7bb1718bee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:30.872700 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c5927-97fa-4490-b685-ba7bb1718bee service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:30.882523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1103/2241] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:45:30.916303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1139/2242] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:30 2022] PUT /identity/v3/projects/fe9cc4682d1a4df8ad1a20f09080f016/users/390a29730a7a401798a8657fca1f0fe9/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:30.924451 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c5927-97fa-4490-b685-ba7bb1718bee service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:30.933583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c5927-97fa-4490-b685-ba7bb1718bee service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:30.999963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3387104-0089-4708-9374-6a7c9f7a6b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:30.999963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3387104-0089-4708-9374-6a7c9f7a6b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:30.999963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3387104-0089-4708-9374-6a7c9f7a6b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:30.999963 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3387104-0089-4708-9374-6a7c9f7a6b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:31.000789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3387104-0089-4708-9374-6a7c9f7a6b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:31.002286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3387104-0089-4708-9374-6a7c9f7a6b99 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:31.013155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1104/2243] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:31.017312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025aa64c-c0ac-4e77-812e-e2db2b31af36 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:31.028599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025aa64c-c0ac-4e77-812e-e2db2b31af36 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:31.033244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c06739-8467-4e4a-aa86-b02b00b13e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:31.033889 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c06739-8467-4e4a-aa86-b02b00b13e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:31.034199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c06739-8467-4e4a-aa86-b02b00b13e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:31.034471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c06739-8467-4e4a-aa86-b02b00b13e51 admin admin] PATH_INFO: `/v3/projects/fe9cc4682d1a4df8ad1a20f09080f016/users/390a29730a7a401798a8657fca1f0fe9/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:31.036422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c06739-8467-4e4a-aa86-b02b00b13e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9cc4682d1a4df8ad1a20f09080f016, user_id=390a29730a7a401798a8657fca1f0fe9, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:31.039160 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c06739-8467-4e4a-aa86-b02b00b13e51 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:31.050208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1140/2244] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:31 2022] PUT /identity/v3/projects/fe9cc4682d1a4df8ad1a20f09080f016/users/390a29730a7a401798a8657fca1f0fe9/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:31.062997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05305261-77fc-428d-9c1d-ad473bc459f3 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:31.062997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05305261-77fc-428d-9c1d-ad473bc459f3 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:31.062997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05305261-77fc-428d-9c1d-ad473bc459f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:31.094006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-05305261-77fc-428d-9c1d-ad473bc459f3 None None] MFA Rules not processed for user `390a29730a7a401798a8657fca1f0fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:31.095328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-05305261-77fc-428d-9c1d-ad473bc459f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:31.108193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c06739-8467-4e4a-aa86-b02b00b13e51 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:31.165505 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1105/2245] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:31.173868 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c06739-8467-4e4a-aa86-b02b00b13e51 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:31.177026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05305261-77fc-428d-9c1d-ad473bc459f3 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:31.178429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-05305261-77fc-428d-9c1d-ad473bc459f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:31.179591 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:31.180219 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:31.180495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:31.180760 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] PATH_INFO: `/v3/users/3f701505e4d24954a01dc8252944320c` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:31.181869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f701505e4d24954a01dc8252944320c)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:31.183923 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:31.223068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:31.237166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05305261-77fc-428d-9c1d-ad473bc459f3 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:31.240918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16589a-2f61-4342-8f36-9205a1b87fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:31.241551 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16589a-2f61-4342-8f36-9205a1b87fdd service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:31.244749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16589a-2f61-4342-8f36-9205a1b87fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:31.244749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16589a-2f61-4342-8f36-9205a1b87fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:31.244749 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4f16589a-2f61-4342-8f36-9205a1b87fdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:31.301054 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16589a-2f61-4342-8f36-9205a1b87fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:31.302443 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16589a-2f61-4342-8f36-9205a1b87fdd service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:31.311391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1106/2246] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:31.398314 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f701505e4d24954a01dc8252944320c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:31.411587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f701505e4d24954a01dc8252944320c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:31.412266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f701505e4d24954a01dc8252944320c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:31.480013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f701505e4d24954a01dc8252944320c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:31.487435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f701505e4d24954a01dc8252944320c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:31.493937 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1141/2247] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:31 2022] DELETE /identity/v3/users/3f701505e4d24954a01dc8252944320c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:31.498552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b402-135e-40dd-9066-9d4523cdcc22 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:31.498843 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b402-135e-40dd-9066-9d4523cdcc22 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:31.499121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b402-135e-40dd-9066-9d4523cdcc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:31.519492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-64d5b402-135e-40dd-9066-9d4523cdcc22 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:31.520708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-64d5b402-135e-40dd-9066-9d4523cdcc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:31.579375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1107/2248] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:31.589149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:31.590572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:31.637274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:31.651222 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed068396-295d-40ee-99f5-e6074afb2296 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:31.652818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ede88a-6170-4a30-841b-a40c79983e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:31.653423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede88a-6170-4a30-841b-a40c79983e38 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:31.653702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede88a-6170-4a30-841b-a40c79983e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:31.653995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ede88a-6170-4a30-841b-a40c79983e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:31.662985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ede88a-6170-4a30-841b-a40c79983e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:31.664420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ede88a-6170-4a30-841b-a40c79983e38 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:31.673662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1142/2249] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:32.409056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5b402-135e-40dd-9066-9d4523cdcc22 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:32.410550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-64d5b402-135e-40dd-9066-9d4523cdcc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:32.425759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5b402-135e-40dd-9066-9d4523cdcc22 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:32.430824 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:32.434763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:32.434763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:32.434763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] PATH_INFO: `/v3/projects/7f1531c48a844620bdbb4c6de446071e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:32.434763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f1531c48a844620bdbb4c6de446071e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:32.435990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:32.567321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f1531c48a844620bdbb4c6de446071e', 'request_id': 'req-bf487e51-78f8-49fa-8eb7-de046b245d92'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:32.573896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f1531c48a844620bdbb4c6de446071e', 'request_id': 'req-bf487e51-78f8-49fa-8eb7-de046b245d92'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:32.580230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f1531c48a844620bdbb4c6de446071e', 'request_id': 'req-bf487e51-78f8-49fa-8eb7-de046b245d92'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:32.669369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] The token cache is being invalidate because project 7f1531c48a844620bdbb4c6de446071e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:32.669585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:32.670307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-bf487e51-78f8-49fa-8eb7-de046b245d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:32.671845 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1108/2250] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:32 2022] DELETE /identity/v3/projects/7f1531c48a844620bdbb4c6de446071e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:32.676654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a0a6b-5d57-4f71-960c-553997a8a184 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:32.677107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a0a6b-5d57-4f71-960c-553997a8a184 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:32.677215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a0a6b-5d57-4f71-960c-553997a8a184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:32.697510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-bf9a0a6b-5d57-4f71-960c-553997a8a184 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:32.698751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bf9a0a6b-5d57-4f71-960c-553997a8a184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:32.759117 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1143/2251] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:32.767199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade194-b777-4243-9dce-53c3ff711507 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:32.767481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade194-b777-4243-9dce-53c3ff711507 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:32.767753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade194-b777-4243-9dce-53c3ff711507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:32.788375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-67ade194-b777-4243-9dce-53c3ff711507 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:32.789697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-67ade194-b777-4243-9dce-53c3ff711507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:32.797706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1109/2252] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:32.802140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a0a6b-5d57-4f71-960c-553997a8a184 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:32.812138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a0a6b-5d57-4f71-960c-553997a8a184 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:32.816695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:32.817352 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:32.817630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:32.817905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:32.824914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:32.824914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:32.824914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:32.833662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1144/2253] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:45:32.837870 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade194-b777-4243-9dce-53c3ff711507 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:32.847266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade194-b777-4243-9dce-53c3ff711507 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:32.852026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83fee6-2e32-4900-b9db-3396cc3cac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:32.852615 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83fee6-2e32-4900-b9db-3396cc3cac29 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:32.852895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83fee6-2e32-4900-b9db-3396cc3cac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:32.853167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83fee6-2e32-4900-b9db-3396cc3cac29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:32.853795 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83fee6-2e32-4900-b9db-3396cc3cac29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:32.855726 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83fee6-2e32-4900-b9db-3396cc3cac29 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:32.865141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:32.963360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1110/2254] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:32 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:32.967619 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:32.985940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf5344-7f1f-4cab-8cd4-c916b10684d3 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.037236 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c443fa1-1733-4863-9d4a-671aba175539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.038048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443fa1-1733-4863-9d4a-671aba175539 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.038335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443fa1-1733-4863-9d4a-671aba175539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.038610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c443fa1-1733-4863-9d4a-671aba175539 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.039254 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c443fa1-1733-4863-9d4a-671aba175539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.041327 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c443fa1-1733-4863-9d4a-671aba175539 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.122774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1145/2255] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:32 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:33.125804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83fee6-2e32-4900-b9db-3396cc3cac29 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.137398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83fee6-2e32-4900-b9db-3396cc3cac29 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.146069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6af715-cbee-4ed4-b26f-47b19470126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.146069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6af715-cbee-4ed4-b26f-47b19470126b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.146069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6af715-cbee-4ed4-b26f-47b19470126b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.146069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6af715-cbee-4ed4-b26f-47b19470126b admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.146482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6af715-cbee-4ed4-b26f-47b19470126b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.146482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6af715-cbee-4ed4-b26f-47b19470126b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.163948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1111/2256] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:33.165117 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c443fa1-1733-4863-9d4a-671aba175539 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.175362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c443fa1-1733-4863-9d4a-671aba175539 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.180589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.181192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.181469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.181739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.182474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.184062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.195050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1146/2257] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:45:33.203622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6af715-cbee-4ed4-b26f-47b19470126b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.209863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6af715-cbee-4ed4-b26f-47b19470126b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.214917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15c7b0-2933-4527-aa69-76e020d716a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.215595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c7b0-2933-4527-aa69-76e020d716a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.215878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c7b0-2933-4527-aa69-76e020d716a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.216155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c7b0-2933-4527-aa69-76e020d716a9 admin admin] PATH_INFO: `/v3/projects/75f911dd7ca84724b66cea1f840c336d/users/9cefe9917fef4d009238a98b326e192e/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.222695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.224479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:33.232025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15c7b0-2933-4527-aa69-76e020d716a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f911dd7ca84724b66cea1f840c336d, user_id=9cefe9917fef4d009238a98b326e192e, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.234706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15c7b0-2933-4527-aa69-76e020d716a9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.282195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d734-a962-4fc5-a06a-a7b35f9df93e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.286017 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.286615 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.286892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.287165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.288908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:33.294341 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:33.310786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1112/2258] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:33 2022] PUT /identity/v3/projects/75f911dd7ca84724b66cea1f840c336d/users/9cefe9917fef4d009238a98b326e192e/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:33.321286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668358a-8b4f-4354-9ca9-0ed5f2d3b03e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.321565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668358a-8b4f-4354-9ca9-0ed5f2d3b03e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.321864 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668358a-8b4f-4354-9ca9-0ed5f2d3b03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.349377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-5668358a-8b4f-4354-9ca9-0ed5f2d3b03e None None] MFA Rules not processed for user `9cefe9917fef4d009238a98b326e192e`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:33.350652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5668358a-8b4f-4354-9ca9-0ed5f2d3b03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:33.351359 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.353309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.362470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1147/2259] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:45:33.416389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1113/2260] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:33.420846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.431892 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5012d-69ce-4125-81ba-3cd55e94d939 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.489564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff95a9-961f-400e-a453-3e2539609dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.490120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff95a9-961f-400e-a453-3e2539609dae admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.494118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff95a9-961f-400e-a453-3e2539609dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.494118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff95a9-961f-400e-a453-3e2539609dae admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.494118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff95a9-961f-400e-a453-3e2539609dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.494118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff95a9-961f-400e-a453-3e2539609dae admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.579205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1148/2261] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:33 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:33.583187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668358a-8b4f-4354-9ca9-0ed5f2d3b03e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.594857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668358a-8b4f-4354-9ca9-0ed5f2d3b03e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.653148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b7573-d6d2-4b59-bfc6-2d364208e726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.653752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b7573-d6d2-4b59-bfc6-2d364208e726 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.654060 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b7573-d6d2-4b59-bfc6-2d364208e726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.654342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b7573-d6d2-4b59-bfc6-2d364208e726 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.654966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b7573-d6d2-4b59-bfc6-2d364208e726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.656974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b7573-d6d2-4b59-bfc6-2d364208e726 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.778612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1114/2262] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:33 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:33.782504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff95a9-961f-400e-a453-3e2539609dae admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.793470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff95a9-961f-400e-a453-3e2539609dae admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.797913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9411f9-bdb7-4514-847b-28ce81482e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.798523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9411f9-bdb7-4514-847b-28ce81482e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.798799 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9411f9-bdb7-4514-847b-28ce81482e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.799066 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9411f9-bdb7-4514-847b-28ce81482e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.799705 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9411f9-bdb7-4514-847b-28ce81482e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.801311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9411f9-bdb7-4514-847b-28ce81482e4e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.812688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1149/2263] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:33.816903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b7573-d6d2-4b59-bfc6-2d364208e726 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.827406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b7573-d6d2-4b59-bfc6-2d364208e726 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:33.832084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c72c2-41c8-44fc-afb8-a1d3b4b9e59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:33.832752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c72c2-41c8-44fc-afb8-a1d3b4b9e59f admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:33.839242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c72c2-41c8-44fc-afb8-a1d3b4b9e59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:33.839242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c72c2-41c8-44fc-afb8-a1d3b4b9e59f admin admin] PATH_INFO: `/v3/projects/3913ba9312184a44a031395a6bb01df2/users/64965735c3fe4f0489e7d2054923d172/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:33.849072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c72c2-41c8-44fc-afb8-a1d3b4b9e59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3913ba9312184a44a031395a6bb01df2, user_id=64965735c3fe4f0489e7d2054923d172, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:33.852439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c72c2-41c8-44fc-afb8-a1d3b4b9e59f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:33.967161 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1115/2264] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:33 2022] PUT /identity/v3/projects/3913ba9312184a44a031395a6bb01df2/users/64965735c3fe4f0489e7d2054923d172/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:33.971216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9411f9-bdb7-4514-847b-28ce81482e4e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:33.982635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9411f9-bdb7-4514-847b-28ce81482e4e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:34.039172 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9179de2-db2c-4e07-84cd-b6561fac21ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:34.039771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9179de2-db2c-4e07-84cd-b6561fac21ca admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:34.040048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9179de2-db2c-4e07-84cd-b6561fac21ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:34.040317 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9179de2-db2c-4e07-84cd-b6561fac21ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:34.040948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9179de2-db2c-4e07-84cd-b6561fac21ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:34.042566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9179de2-db2c-4e07-84cd-b6561fac21ca admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:34.052401 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1150/2265] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:34.056620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c72c2-41c8-44fc-afb8-a1d3b4b9e59f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:34.067290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c72c2-41c8-44fc-afb8-a1d3b4b9e59f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:34.072733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:34.073386 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:34.073662 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:34.073949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] PATH_INFO: `/v3/projects/3913ba9312184a44a031395a6bb01df2/users/64965735c3fe4f0489e7d2054923d172/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:34.076009 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3913ba9312184a44a031395a6bb01df2, user_id=64965735c3fe4f0489e7d2054923d172, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:34.078807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:34.082217 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:34.090842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1116/2266] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:34 2022] PUT /identity/v3/projects/3913ba9312184a44a031395a6bb01df2/users/64965735c3fe4f0489e7d2054923d172/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:34.102380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ac1bb-6cbe-40a9-a772-7c6236e85400 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:34.102626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ac1bb-6cbe-40a9-a772-7c6236e85400 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:34.102897 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ac1bb-6cbe-40a9-a772-7c6236e85400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:34.130773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-200ac1bb-6cbe-40a9-a772-7c6236e85400 None None] MFA Rules not processed for user `64965735c3fe4f0489e7d2054923d172`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:34.132057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-200ac1bb-6cbe-40a9-a772-7c6236e85400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:34.136495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:34.199807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1151/2267] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:34.212364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:34.214215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:34.271075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76add9-b46c-4987-9aca-c8c31e3f5175 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:34.275301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346cf9-2aa2-4e62-b673-bb7d86420ad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:34.275890 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346cf9-2aa2-4e62-b673-bb7d86420ad0 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:34.276165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346cf9-2aa2-4e62-b673-bb7d86420ad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:34.276434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346cf9-2aa2-4e62-b673-bb7d86420ad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:34.285361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6346cf9-2aa2-4e62-b673-bb7d86420ad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:34.286861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6346cf9-2aa2-4e62-b673-bb7d86420ad0 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:34.309175 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1117/2268] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:34.841315 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ac1bb-6cbe-40a9-a772-7c6236e85400 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:34.844728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ac1bb-6cbe-40a9-a772-7c6236e85400 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:34.848360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69fa5e-1a87-433b-857e-25a59fa0867d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:34.848950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69fa5e-1a87-433b-857e-25a59fa0867d service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:34.849252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69fa5e-1a87-433b-857e-25a59fa0867d service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:34.849528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69fa5e-1a87-433b-857e-25a59fa0867d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:34.851320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1e69fa5e-1a87-433b-857e-25a59fa0867d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:34.919230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69fa5e-1a87-433b-857e-25a59fa0867d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:34.920641 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69fa5e-1a87-433b-857e-25a59fa0867d service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:34.930241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1152/2269] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:35.223740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346cf9-2aa2-4e62-b673-bb7d86420ad0 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:35.291243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346cf9-2aa2-4e62-b673-bb7d86420ad0 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:35.296691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:35.297534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:35.297815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:35.298123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] PATH_INFO: `/v3/users/9cefe9917fef4d009238a98b326e192e` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:35.299247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cefe9917fef4d009238a98b326e192e)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:35.301149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:35.482746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cefe9917fef4d009238a98b326e192e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:35.495134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cefe9917fef4d009238a98b326e192e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:35.565589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cefe9917fef4d009238a98b326e192e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:35.566256 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cefe9917fef4d009238a98b326e192e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:35.573743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cefe9917fef4d009238a98b326e192e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:35.579801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1118/2270] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:35 2022] DELETE /identity/v3/users/9cefe9917fef4d009238a98b326e192e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:35.584362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ae336-f1a6-41cc-ad1a-b19645d24bd4 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:35.584644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ae336-f1a6-41cc-ad1a-b19645d24bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:35.584915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ae336-f1a6-41cc-ad1a-b19645d24bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:35.603950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-883ae336-f1a6-41cc-ad1a-b19645d24bd4 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:35.605153 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-883ae336-f1a6-41cc-ad1a-b19645d24bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:35.662451 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1153/2271] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:35.672895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:35.674517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:35.695263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:35.731725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf739c-836e-4a10-856e-a3f4778221d5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:35.735501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45ce0f-7108-4413-b1dc-d8929e199e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:35.736040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45ce0f-7108-4413-b1dc-d8929e199e82 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:35.736321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45ce0f-7108-4413-b1dc-d8929e199e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:35.736595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45ce0f-7108-4413-b1dc-d8929e199e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:35.746155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45ce0f-7108-4413-b1dc-d8929e199e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:35.747577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45ce0f-7108-4413-b1dc-d8929e199e82 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:35.757464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1119/2272] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:45:36.156911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ae336-f1a6-41cc-ad1a-b19645d24bd4 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:36.158473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-883ae336-f1a6-41cc-ad1a-b19645d24bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:36.172572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ae336-f1a6-41cc-ad1a-b19645d24bd4 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:36.177394 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:36.178004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:36.178293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:36.178567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] PATH_INFO: `/v3/projects/75f911dd7ca84724b66cea1f840c336d` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:36.179684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f911dd7ca84724b66cea1f840c336d)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:36.181648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:36.276575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:36.292939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f911dd7ca84724b66cea1f840c336d', 'request_id': 'req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.305489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f911dd7ca84724b66cea1f840c336d', 'request_id': 'req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.305489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f911dd7ca84724b66cea1f840c336d', 'request_id': 'req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.371721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] The token cache is being invalidate because project 75f911dd7ca84724b66cea1f840c336d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:36.372012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.372635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.374135 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1154/2273] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:36 2022] DELETE /identity/v3/projects/75f911dd7ca84724b66cea1f840c336d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:36.378782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45ce0f-7108-4413-b1dc-d8929e199e82 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:36.389539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45ce0f-7108-4413-b1dc-d8929e199e82 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:36.446718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:36.447350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:36.447633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:36.447911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] PATH_INFO: `/v3/users/64965735c3fe4f0489e7d2054923d172` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:36.449065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] RBAC: Authorizing `identity:delete_user(user_id=64965735c3fe4f0489e7d2054923d172)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:36.451023 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:36.672316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64965735c3fe4f0489e7d2054923d172'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.684836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64965735c3fe4f0489e7d2054923d172'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.727812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64965735c3fe4f0489e7d2054923d172'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.728484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64965735c3fe4f0489e7d2054923d172'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.736051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29219a3f-de74-4f9c-bb23-a7612815264c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64965735c3fe4f0489e7d2054923d172'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:36.742866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1120/2274] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:36 2022] DELETE /identity/v3/users/64965735c3fe4f0489e7d2054923d172 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:37.144565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:37.150076 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:37.162176 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46aed7-b40a-465d-afc8-d9c3fbfb9e7a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:37.221556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:37.222248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.222516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.222791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] PATH_INFO: `/v3/projects/3913ba9312184a44a031395a6bb01df2` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.224005 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3913ba9312184a44a031395a6bb01df2)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:37.225901 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:37.356081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3913ba9312184a44a031395a6bb01df2', 'request_id': 'req-1853d8a4-af1c-42ad-b42d-8d0423f824cc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:37.361585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3913ba9312184a44a031395a6bb01df2', 'request_id': 'req-1853d8a4-af1c-42ad-b42d-8d0423f824cc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:37.368240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3913ba9312184a44a031395a6bb01df2', 'request_id': 'req-1853d8a4-af1c-42ad-b42d-8d0423f824cc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:37.464014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] The token cache is being invalidate because project 3913ba9312184a44a031395a6bb01df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:37.464297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:37.464912 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-1853d8a4-af1c-42ad-b42d-8d0423f824cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:37.466423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1155/2275] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:37 2022] DELETE /identity/v3/projects/3913ba9312184a44a031395a6bb01df2 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:37.471321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc71e-b9ab-4d12-a189-2022d6b42f60 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.471816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc71e-b9ab-4d12-a189-2022d6b42f60 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.471912 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163bc71e-b9ab-4d12-a189-2022d6b42f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.491986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-163bc71e-b9ab-4d12-a189-2022d6b42f60 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:37.493180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-163bc71e-b9ab-4d12-a189-2022d6b42f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:37.526517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:37.551834 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1121/2276] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:37.559687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1b20-d9ac-4281-8fed-f7e83bae0cf8 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.559972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1b20-d9ac-4281-8fed-f7e83bae0cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.560276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1b20-d9ac-4281-8fed-f7e83bae0cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.580348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d9cb1b20-d9ac-4281-8fed-f7e83bae0cf8 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:37.581555 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d9cb1b20-d9ac-4281-8fed-f7e83bae0cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:37.590373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1156/2277] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:37.594474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bc71e-b9ab-4d12-a189-2022d6b42f60 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:37.604815 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163bc71e-b9ab-4d12-a189-2022d6b42f60 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:37.612068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:37.612068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.612068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.612068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.612477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:37.612477 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:37.613186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:37.624735 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1122/2278] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:45:37.628927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb1b20-d9ac-4281-8fed-f7e83bae0cf8 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:37.638122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb1b20-d9ac-4281-8fed-f7e83bae0cf8 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:37.642906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76123e1b-924a-480a-9b04-e2c5f34433c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:37.643496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76123e1b-924a-480a-9b04-e2c5f34433c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.643779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76123e1b-924a-480a-9b04-e2c5f34433c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.644052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76123e1b-924a-480a-9b04-e2c5f34433c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.644686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76123e1b-924a-480a-9b04-e2c5f34433c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:37.646633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76123e1b-924a-480a-9b04-e2c5f34433c1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:37.710299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1157/2279] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:37 2022] POST /identity/v3/projects => generated 363 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:45:37.714935 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:37.724968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa16e0-677d-4ba7-82a0-ef88e2e6b1fa admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:37.783463 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9549e6-de8b-4d64-8f09-15a19a537125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:37.784064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9549e6-de8b-4d64-8f09-15a19a537125 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.784345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9549e6-de8b-4d64-8f09-15a19a537125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.784618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9549e6-de8b-4d64-8f09-15a19a537125 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.785248 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9549e6-de8b-4d64-8f09-15a19a537125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:37.787178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9549e6-de8b-4d64-8f09-15a19a537125 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:37.860830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1123/2280] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:37 2022] POST /identity/v3/users => generated 282 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:45:37.867539 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76123e1b-924a-480a-9b04-e2c5f34433c1 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:37.879419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76123e1b-924a-480a-9b04-e2c5f34433c1 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:37.885009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163cef7-d2e0-46bd-bee4-201e1217de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:37.885619 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163cef7-d2e0-46bd-bee4-201e1217de26 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.899469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163cef7-d2e0-46bd-bee4-201e1217de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.899469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163cef7-d2e0-46bd-bee4-201e1217de26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.899469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163cef7-d2e0-46bd-bee4-201e1217de26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:37.899469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163cef7-d2e0-46bd-bee4-201e1217de26 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:37.899469 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1158/2281] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:37.903178 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9549e6-de8b-4d64-8f09-15a19a537125 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:37.913281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9549e6-de8b-4d64-8f09-15a19a537125 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:37.917918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995f40-a484-4b0a-8c00-c2549d032173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:37.918563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995f40-a484-4b0a-8c00-c2549d032173 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.918838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995f40-a484-4b0a-8c00-c2549d032173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.919187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995f40-a484-4b0a-8c00-c2549d032173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.919742 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c995f40-a484-4b0a-8c00-c2549d032173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:37.921311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c995f40-a484-4b0a-8c00-c2549d032173 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:37.931984 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1124/2282] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:37.936370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163cef7-d2e0-46bd-bee4-201e1217de26 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:37.949030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163cef7-d2e0-46bd-bee4-201e1217de26 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:37.954217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20983244-4d41-4389-86fb-78e17e5f755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:37.958471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20983244-4d41-4389-86fb-78e17e5f755f admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.958471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20983244-4d41-4389-86fb-78e17e5f755f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.958471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20983244-4d41-4389-86fb-78e17e5f755f admin admin] PATH_INFO: `/v3/projects/a30be59c78bd42ceb72b243e9b002fce/users/acfee12303e748dfa661089ef43345bc/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.959008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:37.971623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20983244-4d41-4389-86fb-78e17e5f755f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a30be59c78bd42ceb72b243e9b002fce, user_id=acfee12303e748dfa661089ef43345bc, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:37.974429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20983244-4d41-4389-86fb-78e17e5f755f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:38.083517 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1159/2283] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:37 2022] PUT /identity/v3/projects/a30be59c78bd42ceb72b243e9b002fce/users/acfee12303e748dfa661089ef43345bc/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:38.094727 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baee0ec-d258-435a-8dde-f6dae9d0065e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:38.095207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baee0ec-d258-435a-8dde-f6dae9d0065e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:38.095838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baee0ec-d258-435a-8dde-f6dae9d0065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:38.129970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-7baee0ec-d258-435a-8dde-f6dae9d0065e None None] MFA Rules not processed for user `acfee12303e748dfa661089ef43345bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:38.129970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7baee0ec-d258-435a-8dde-f6dae9d0065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:38.197732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1125/2284] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:38.202355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20983244-4d41-4389-86fb-78e17e5f755f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:38.265715 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20983244-4d41-4389-86fb-78e17e5f755f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:38.271852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46b23-0e23-435b-9461-f25a90277e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:38.272781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46b23-0e23-435b-9461-f25a90277e3a admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:38.273072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46b23-0e23-435b-9461-f25a90277e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:38.273556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46b23-0e23-435b-9461-f25a90277e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:38.274559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc46b23-0e23-435b-9461-f25a90277e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:38.277087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc46b23-0e23-435b-9461-f25a90277e3a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:38.375590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1160/2285] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:38 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:38.381220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baee0ec-d258-435a-8dde-f6dae9d0065e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:38.392808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baee0ec-d258-435a-8dde-f6dae9d0065e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:38.426215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:38.455377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140f1af-b0f4-4188-ac7d-43129a888a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:38.456204 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f1af-b0f4-4188-ac7d-43129a888a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:38.456687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f1af-b0f4-4188-ac7d-43129a888a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:38.456976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140f1af-b0f4-4188-ac7d-43129a888a96 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:38.457878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140f1af-b0f4-4188-ac7d-43129a888a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:38.460221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140f1af-b0f4-4188-ac7d-43129a888a96 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:38.574927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1126/2286] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:38 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:38.576173 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46b23-0e23-435b-9461-f25a90277e3a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:38.586977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46b23-0e23-435b-9461-f25a90277e3a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:38.591811 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533603c2-42a6-4df1-9985-eec1fed06057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:38.592411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533603c2-42a6-4df1-9985-eec1fed06057 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:38.592689 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533603c2-42a6-4df1-9985-eec1fed06057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:38.592998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533603c2-42a6-4df1-9985-eec1fed06057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:38.593795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533603c2-42a6-4df1-9985-eec1fed06057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:38.595459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533603c2-42a6-4df1-9985-eec1fed06057 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:38.607091 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1161/2287] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:38.624417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140f1af-b0f4-4188-ac7d-43129a888a96 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:38.634928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140f1af-b0f4-4188-ac7d-43129a888a96 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:38.639554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b2282-25c9-4e5b-bcbd-55f6f0dd0854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:38.640155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b2282-25c9-4e5b-bcbd-55f6f0dd0854 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:38.640434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b2282-25c9-4e5b-bcbd-55f6f0dd0854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:38.640708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b2282-25c9-4e5b-bcbd-55f6f0dd0854 admin admin] PATH_INFO: `/v3/projects/af3906ab3d8341b0afae713f243281e7/users/316cf5c7eefb417aa206727960656793/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:38.655230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b2282-25c9-4e5b-bcbd-55f6f0dd0854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3906ab3d8341b0afae713f243281e7, user_id=316cf5c7eefb417aa206727960656793, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:38.657875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b2282-25c9-4e5b-bcbd-55f6f0dd0854 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:38.760867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1127/2288] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:38 2022] PUT /identity/v3/projects/af3906ab3d8341b0afae713f243281e7/users/316cf5c7eefb417aa206727960656793/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:38.770354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533603c2-42a6-4df1-9985-eec1fed06057 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:38.781244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533603c2-42a6-4df1-9985-eec1fed06057 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:38.838023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be1727-cd2c-438a-a789-447ea1888e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:38.838655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be1727-cd2c-438a-a789-447ea1888e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:38.838933 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be1727-cd2c-438a-a789-447ea1888e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:38.839208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be1727-cd2c-438a-a789-447ea1888e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:38.839879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be1727-cd2c-438a-a789-447ea1888e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:38.841515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be1727-cd2c-438a-a789-447ea1888e89 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:38.851257 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1162/2289] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:38.856252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b2282-25c9-4e5b-bcbd-55f6f0dd0854 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:38.866666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b2282-25c9-4e5b-bcbd-55f6f0dd0854 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:38.872181 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:38.872982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:38.873262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:38.873567 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] PATH_INFO: `/v3/projects/af3906ab3d8341b0afae713f243281e7/users/316cf5c7eefb417aa206727960656793/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:38.875796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3906ab3d8341b0afae713f243281e7, user_id=316cf5c7eefb417aa206727960656793, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:38.879481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:38.889031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1128/2290] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:38 2022] PUT /identity/v3/projects/af3906ab3d8341b0afae713f243281e7/users/316cf5c7eefb417aa206727960656793/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:38.901164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905367e-e565-4ce3-822f-eda5c88dd4c7 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:38.901449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905367e-e565-4ce3-822f-eda5c88dd4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:38.901726 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905367e-e565-4ce3-822f-eda5c88dd4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:38.931204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b905367e-e565-4ce3-822f-eda5c88dd4c7 None None] MFA Rules not processed for user `316cf5c7eefb417aa206727960656793`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:38.932421 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b905367e-e565-4ce3-822f-eda5c88dd4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:38.977097 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:39.002412 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1163/2291] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:39.013577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:39.015286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:39.076705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a4fe5-327a-4882-8b84-df6279104667 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:39.080609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7cc5c-b510-45e8-b185-33f24b17d6b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:39.081213 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7cc5c-b510-45e8-b185-33f24b17d6b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:39.081492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7cc5c-b510-45e8-b185-33f24b17d6b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:39.081763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7cc5c-b510-45e8-b185-33f24b17d6b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:39.091769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7cc5c-b510-45e8-b185-33f24b17d6b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:39.093205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7cc5c-b510-45e8-b185-33f24b17d6b1 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:39.103055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1129/2292] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:39.585587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905367e-e565-4ce3-822f-eda5c88dd4c7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:39.587727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b905367e-e565-4ce3-822f-eda5c88dd4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:39.659938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905367e-e565-4ce3-822f-eda5c88dd4c7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:39.664614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:39.665304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:39.669276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:39.669276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] PATH_INFO: `/v3/users/93e75a5e2554402c9f15d4db876a3560` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:39.669276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e75a5e2554402c9f15d4db876a3560)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:39.669276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:39.872049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e75a5e2554402c9f15d4db876a3560'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:39.885249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e75a5e2554402c9f15d4db876a3560'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:39.886392 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e75a5e2554402c9f15d4db876a3560'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:39.983927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e75a5e2554402c9f15d4db876a3560'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:39.989461 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7cc5c-b510-45e8-b185-33f24b17d6b1 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:39.997473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-7da0755e-97cc-46f0-9d9e-cbdbe38181e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e75a5e2554402c9f15d4db876a3560'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:40.003208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7cc5c-b510-45e8-b185-33f24b17d6b1 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:40.004234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1164/2293] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:39 2022] DELETE /identity/v3/users/93e75a5e2554402c9f15d4db876a3560 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:40.009375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98581-ef8e-4aa3-a900-aef22421f06b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:40.009761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98581-ef8e-4aa3-a900-aef22421f06b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:40.010130 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98581-ef8e-4aa3-a900-aef22421f06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:40.039614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-29e98581-ef8e-4aa3-a900-aef22421f06b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:40.040858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-29e98581-ef8e-4aa3-a900-aef22421f06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:40.073712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:40.074609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:40.075016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:40.075307 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] PATH_INFO: `/v3/users/acfee12303e748dfa661089ef43345bc` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:40.076457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfee12303e748dfa661089ef43345bc)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:40.078435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:40.112397 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:40.121596 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1165/2294] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:40.131804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e98581-ef8e-4aa3-a900-aef22421f06b None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:40.133220 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-29e98581-ef8e-4aa3-a900-aef22421f06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:40.192800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e98581-ef8e-4aa3-a900-aef22421f06b None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:40.196600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8d9fc-d64f-4092-9b75-430141a2fa72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:40.197204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d9fc-d64f-4092-9b75-430141a2fa72 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:40.197483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d9fc-d64f-4092-9b75-430141a2fa72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:40.233965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8d9fc-d64f-4092-9b75-430141a2fa72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:40.233965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8d9fc-d64f-4092-9b75-430141a2fa72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:40.233965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8d9fc-d64f-4092-9b75-430141a2fa72 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:40.233965 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1166/2295] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:45:40.290675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:40.302034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfee12303e748dfa661089ef43345bc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:40.316380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfee12303e748dfa661089ef43345bc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:40.430740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfee12303e748dfa661089ef43345bc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:40.434095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfee12303e748dfa661089ef43345bc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:40.441976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfee12303e748dfa661089ef43345bc'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:40.448659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1130/2296] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:39 2022] DELETE /identity/v3/users/acfee12303e748dfa661089ef43345bc => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:40.453134 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507e44f-51eb-421e-b11f-75e061f4e643 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:40.453423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507e44f-51eb-421e-b11f-75e061f4e643 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:40.453721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507e44f-51eb-421e-b11f-75e061f4e643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:40.473512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-3507e44f-51eb-421e-b11f-75e061f4e643 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:40.474907 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3507e44f-51eb-421e-b11f-75e061f4e643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:40.535543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1167/2297] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:40.546262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:40.547702 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:40.616075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dec75-9b0d-4565-a055-fb7b55c2a98b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:40.619903 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d015-8eab-4ff4-924b-848ac3093159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:40.620539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d015-8eab-4ff4-924b-848ac3093159 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:40.620816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d015-8eab-4ff4-924b-848ac3093159 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:40.621093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d015-8eab-4ff4-924b-848ac3093159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:40.635590 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d015-8eab-4ff4-924b-848ac3093159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:40.637024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d015-8eab-4ff4-924b-848ac3093159 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:40.646800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1131/2298] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:45:40.654516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507e44f-51eb-421e-b11f-75e061f4e643 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:40.656429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3507e44f-51eb-421e-b11f-75e061f4e643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:40.670677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507e44f-51eb-421e-b11f-75e061f4e643 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:40.681852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:40.682660 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:40.691683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:40.691683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] PATH_INFO: `/v3/projects/cb20ae49886b4a97a08c8dd5b60c088c` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:40.691683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb20ae49886b4a97a08c8dd5b60c088c)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:40.691683 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:40.870313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb20ae49886b4a97a08c8dd5b60c088c', 'request_id': 'req-d2a71e29-5747-445a-8255-9b3f30bbd41d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:40.887019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb20ae49886b4a97a08c8dd5b60c088c', 'request_id': 'req-d2a71e29-5747-445a-8255-9b3f30bbd41d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:40.895604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb20ae49886b4a97a08c8dd5b60c088c', 'request_id': 'req-d2a71e29-5747-445a-8255-9b3f30bbd41d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.013845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] The token cache is being invalidate because project cb20ae49886b4a97a08c8dd5b60c088c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:41.014185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.027864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.035250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1168/2299] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:40 2022] DELETE /identity/v3/projects/cb20ae49886b4a97a08c8dd5b60c088c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:41.039381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d015-8eab-4ff4-924b-848ac3093159 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:41.049799 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d015-8eab-4ff4-924b-848ac3093159 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:41.123766 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:41.125224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:41.125720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:41.126575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:41.127149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:41.127675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] PATH_INFO: `/v3/users/120c62404f544d09a43e36078f01bd3b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:41.129237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=120c62404f544d09a43e36078f01bd3b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:41.141788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:41.142029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a71e29-5747-445a-8255-9b3f30bbd41d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:41.151523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:41.152163 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:41.152438 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:41.152708 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] PATH_INFO: `/v3/projects/a30be59c78bd42ceb72b243e9b002fce` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:41.155899 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a30be59c78bd42ceb72b243e9b002fce)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:41.171648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:41.216290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:41.396938 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '120c62404f544d09a43e36078f01bd3b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.399409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a30be59c78bd42ceb72b243e9b002fce', 'request_id': 'req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.405018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:41.407898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a30be59c78bd42ceb72b243e9b002fce', 'request_id': 'req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.415235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '120c62404f544d09a43e36078f01bd3b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.419389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a30be59c78bd42ceb72b243e9b002fce', 'request_id': 'req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.463797 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '120c62404f544d09a43e36078f01bd3b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.464482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '120c62404f544d09a43e36078f01bd3b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.472975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '120c62404f544d09a43e36078f01bd3b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.480482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1132/2300] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:41 2022] DELETE /identity/v3/users/120c62404f544d09a43e36078f01bd3b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:41.522526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] The token cache is being invalidate because project a30be59c78bd42ceb72b243e9b002fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:41.522992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.523788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.525835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1169/2301] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:41 2022] DELETE /identity/v3/projects/a30be59c78bd42ceb72b243e9b002fce => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:41.529737 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:41.532095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:41.609180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ccdc2-aefc-4d32-86bc-203c25cdca28 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:41.613919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:41.614574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:41.614854 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:41.615128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] PATH_INFO: `/v3/users/316cf5c7eefb417aa206727960656793` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:41.616283 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=316cf5c7eefb417aa206727960656793)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:41.618878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:41.840036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316cf5c7eefb417aa206727960656793'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.854587 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316cf5c7eefb417aa206727960656793'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.926331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316cf5c7eefb417aa206727960656793'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.926907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316cf5c7eefb417aa206727960656793'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.935371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:41.936818 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:41.951275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316cf5c7eefb417aa206727960656793'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:41.957653 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1133/2302] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:41 2022] DELETE /identity/v3/users/316cf5c7eefb417aa206727960656793 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:42.026183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a7932-7e7d-46cf-867b-d594a8cc7e0d admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:42.118085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:42.118747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:42.119022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:42.119291 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] PATH_INFO: `/v3/projects/0d9cf0a58e0d4667b794ced6ef704cab` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:42.120494 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d9cf0a58e0d4667b794ced6ef704cab)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:42.122477 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:42.305816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d9cf0a58e0d4667b794ced6ef704cab', 'request_id': 'req-3a32efe6-815c-47a1-b72c-d355df45dc66'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.311824 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d9cf0a58e0d4667b794ced6ef704cab', 'request_id': 'req-3a32efe6-815c-47a1-b72c-d355df45dc66'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.313086 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:42.318465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d9cf0a58e0d4667b794ced6ef704cab', 'request_id': 'req-3a32efe6-815c-47a1-b72c-d355df45dc66'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.417155 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] The token cache is being invalidate because project 0d9cf0a58e0d4667b794ced6ef704cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:42.417446 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.418105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.419718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1170/2303] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:41 2022] DELETE /identity/v3/projects/0d9cf0a58e0d4667b794ced6ef704cab => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:42.424855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:42.426345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:42.440014 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206416d2-18f6-4b37-a3f4-e7d9b283a693 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:42.486909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:42.496381 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:42.497024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:42.497342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:42.497564 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] PATH_INFO: `/v3/users/390a29730a7a401798a8657fca1f0fe9` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:42.498732 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] RBAC: Authorizing `identity:delete_user(user_id=390a29730a7a401798a8657fca1f0fe9)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:42.500894 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:42.533779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:42.535229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:42.549015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32efe6-815c-47a1-b72c-d355df45dc66 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:42.553822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:42.555162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:42.555770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:42.556040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] PATH_INFO: `/v3/projects/af3906ab3d8341b0afae713f243281e7` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:42.557178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3906ab3d8341b0afae713f243281e7)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:42.559187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:42.752510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3906ab3d8341b0afae713f243281e7', 'request_id': 'req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.752790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390a29730a7a401798a8657fca1f0fe9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.758973 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3906ab3d8341b0afae713f243281e7', 'request_id': 'req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.765777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390a29730a7a401798a8657fca1f0fe9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.765971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3906ab3d8341b0afae713f243281e7', 'request_id': 'req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.846466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] The token cache is being invalidate because project af3906ab3d8341b0afae713f243281e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:42.846680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.847403 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-fa65f2b2-a76b-4f90-afe1-da5aa8a3a371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.848900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1171/2304] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:42 2022] DELETE /identity/v3/projects/af3906ab3d8341b0afae713f243281e7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:42.853879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91def9e-8245-4946-b504-93f9c38409ec None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:42.854285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91def9e-8245-4946-b504-93f9c38409ec None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:42.854592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91def9e-8245-4946-b504-93f9c38409ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:42.874715 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c91def9e-8245-4946-b504-93f9c38409ec None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:42.876204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c91def9e-8245-4946-b504-93f9c38409ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:42.898558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390a29730a7a401798a8657fca1f0fe9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.899210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390a29730a7a401798a8657fca1f0fe9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.906869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7abae9fa-39d9-411e-9797-ad1992abe14c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390a29730a7a401798a8657fca1f0fe9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:42.912724 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1134/2305] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:42 2022] DELETE /identity/v3/users/390a29730a7a401798a8657fca1f0fe9 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:42.935562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1172/2306] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:42.943635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c1f76-09e4-4490-9de9-43ce523486a7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:42.943927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c1f76-09e4-4490-9de9-43ce523486a7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:42.944282 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c1f76-09e4-4490-9de9-43ce523486a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:42.964930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-613c1f76-09e4-4490-9de9-43ce523486a7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:42.966235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-613c1f76-09e4-4490-9de9-43ce523486a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:43.026730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1135/2307] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:43.031166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91def9e-8245-4946-b504-93f9c38409ec None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.041524 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91def9e-8245-4946-b504-93f9c38409ec None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.046610 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.047416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.048462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.048791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.049450 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:43.049785 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.051640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.064925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1173/2308] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:45:43.071237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c1f76-09e4-4490-9de9-43ce523486a7 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.080625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c1f76-09e4-4490-9de9-43ce523486a7 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.085422 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0fe16-3958-416e-ba1b-96bbfe6217cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.086208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0fe16-3958-416e-ba1b-96bbfe6217cb admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.086489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0fe16-3958-416e-ba1b-96bbfe6217cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.088804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0fe16-3958-416e-ba1b-96bbfe6217cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.088804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0fe16-3958-416e-ba1b-96bbfe6217cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.089347 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0fe16-3958-416e-ba1b-96bbfe6217cb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.163274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1136/2309] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:43 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:43.167876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.177843 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf08af8-36cc-4722-8e78-2e059e98d5f5 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.256898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:43.262954 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.263653 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.263934 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.264230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.264865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.266898 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.345288 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1174/2310] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:43 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:43.349416 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0fe16-3958-416e-ba1b-96bbfe6217cb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.350769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.353741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:43.361407 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0fe16-3958-416e-ba1b-96bbfe6217cb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.366634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99939ed4-9fe8-40ce-bdea-aa2932f4c675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.367172 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31133a36-bb7d-49ad-ad26-cdff89fe4705 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.367400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939ed4-9fe8-40ce-bdea-aa2932f4c675 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.368249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939ed4-9fe8-40ce-bdea-aa2932f4c675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.368249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939ed4-9fe8-40ce-bdea-aa2932f4c675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.368988 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99939ed4-9fe8-40ce-bdea-aa2932f4c675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.370762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99939ed4-9fe8-40ce-bdea-aa2932f4c675 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.380120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.380844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.381216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.381549 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] PATH_INFO: `/v3/projects/fe9cc4682d1a4df8ad1a20f09080f016` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.382807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9cc4682d1a4df8ad1a20f09080f016)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.383986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1137/2311] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:43.386633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.389844 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99939ed4-9fe8-40ce-bdea-aa2932f4c675 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.404953 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99939ed4-9fe8-40ce-bdea-aa2932f4c675 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.410995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da040ec-a701-470c-a8ca-7bbd910b56b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.411716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da040ec-a701-470c-a8ca-7bbd910b56b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.412626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da040ec-a701-470c-a8ca-7bbd910b56b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.412626 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da040ec-a701-470c-a8ca-7bbd910b56b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.413291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da040ec-a701-470c-a8ca-7bbd910b56b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.416316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da040ec-a701-470c-a8ca-7bbd910b56b0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.430135 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1138/2312] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:43.431204 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da040ec-a701-470c-a8ca-7bbd910b56b0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.441802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da040ec-a701-470c-a8ca-7bbd910b56b0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.447002 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba787a-d004-47d8-ab6a-706f3408e9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.447705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba787a-d004-47d8-ab6a-706f3408e9cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.448050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba787a-d004-47d8-ab6a-706f3408e9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.448389 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba787a-d004-47d8-ab6a-706f3408e9cf admin admin] PATH_INFO: `/v3/projects/dbd890d552c442dcb9aa00a2fca42818/users/e0ed51c8873c47c388da4968ccf99ee0/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.464128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba787a-d004-47d8-ab6a-706f3408e9cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd890d552c442dcb9aa00a2fca42818, user_id=e0ed51c8873c47c388da4968ccf99ee0, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.467383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba787a-d004-47d8-ab6a-706f3408e9cf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.581907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1139/2313] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:43 2022] PUT /identity/v3/projects/dbd890d552c442dcb9aa00a2fca42818/users/e0ed51c8873c47c388da4968ccf99ee0/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:43.596731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9cc4682d1a4df8ad1a20f09080f016', 'request_id': 'req-a26d499d-d783-4f63-90fd-7c9835fa580e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:43.599359 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c8f4-d92f-4767-a68c-868f4a15d1ec None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.599663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c8f4-d92f-4767-a68c-868f4a15d1ec None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.600218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8c8f4-d92f-4767-a68c-868f4a15d1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.605768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9cc4682d1a4df8ad1a20f09080f016', 'request_id': 'req-a26d499d-d783-4f63-90fd-7c9835fa580e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:43.610162 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:43.619116 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9cc4682d1a4df8ad1a20f09080f016', 'request_id': 'req-a26d499d-d783-4f63-90fd-7c9835fa580e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:43.635061 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-1bc8c8f4-d92f-4767-a68c-868f4a15d1ec None None] MFA Rules not processed for user `e0ed51c8873c47c388da4968ccf99ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:43.636271 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1bc8c8f4-d92f-4767-a68c-868f4a15d1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:43.698497 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] The token cache is being invalidate because project fe9cc4682d1a4df8ad1a20f09080f016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:43.698776 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:43.699425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a26d499d-d783-4f63-90fd-7c9835fa580e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:43.704396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1175/2314] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:43 2022] DELETE /identity/v3/projects/fe9cc4682d1a4df8ad1a20f09080f016 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:43.705727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c747db8-3b91-45c9-858e-89a7cf41c10b None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.706100 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c747db8-3b91-45c9-858e-89a7cf41c10b None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.706373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c747db8-3b91-45c9-858e-89a7cf41c10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.711970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1140/2315] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:43.712981 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8c8f4-d92f-4767-a68c-868f4a15d1ec None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.734651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-1c747db8-3b91-45c9-858e-89a7cf41c10b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:43.736388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1c747db8-3b91-45c9-858e-89a7cf41c10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:43.784919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8c8f4-d92f-4767-a68c-868f4a15d1ec None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.791831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.792579 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.792937 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.793208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.793841 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.796529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.807131 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1176/2316] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:43.814479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d9daf-538e-4380-ad30-e6d862c04726 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.814855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d9daf-538e-4380-ad30-e6d862c04726 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.815290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d9daf-538e-4380-ad30-e6d862c04726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.837969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ed8d9daf-538e-4380-ad30-e6d862c04726 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:43.839881 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ed8d9daf-538e-4380-ad30-e6d862c04726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:43.848600 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1177/2317] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:43.852574 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d9daf-538e-4380-ad30-e6d862c04726 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.862101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d9daf-538e-4380-ad30-e6d862c04726 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.866855 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.867553 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.867830 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.868107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.868780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:43.869083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.870859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.885900 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1178/2318] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:45:43.886125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1141/2319] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:43 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:45:43.893083 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.893729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:43.896229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:43.904473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91fe1b-a989-4f02-921e-ed0ff562380a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.963730 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:43.972190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e28f8-5d11-453b-b8c9-6f100c3a6faa admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:43.973979 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c335a-3570-4465-b6da-03e3abe8cc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.974581 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c335a-3570-4465-b6da-03e3abe8cc95 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.974859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c335a-3570-4465-b6da-03e3abe8cc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.975127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c335a-3570-4465-b6da-03e3abe8cc95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.975781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c335a-3570-4465-b6da-03e3abe8cc95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.977024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9b8e-e584-40d4-8b53-28ff09046c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:43.977641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9b8e-e584-40d4-8b53-28ff09046c0a admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:43.977770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c335a-3570-4465-b6da-03e3abe8cc95 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:43.977918 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9b8e-e584-40d4-8b53-28ff09046c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:43.978221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9b8e-e584-40d4-8b53-28ff09046c0a admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:43.978846 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf9b8e-e584-40d4-8b53-28ff09046c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:43.982004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf9b8e-e584-40d4-8b53-28ff09046c0a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.099812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1142/2320] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:43 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:44.102606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1179/2321] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:43 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:44.103773 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9b8e-e584-40d4-8b53-28ff09046c0a admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.106103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c335a-3570-4465-b6da-03e3abe8cc95 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.114887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9b8e-e584-40d4-8b53-28ff09046c0a admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.119438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:44.180280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c335a-3570-4465-b6da-03e3abe8cc95 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.186449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9855af-28ce-40ce-8c83-778ed9a6fafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.187168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9855af-28ce-40ce-8c83-778ed9a6fafe admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.187590 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9855af-28ce-40ce-8c83-778ed9a6fafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.187996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9855af-28ce-40ce-8c83-778ed9a6fafe admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.188629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9855af-28ce-40ce-8c83-778ed9a6fafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.190673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9855af-28ce-40ce-8c83-778ed9a6fafe admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.192210 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fe5c4-c4af-42aa-b46e-3eb8d70ae479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.192839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fe5c4-c4af-42aa-b46e-3eb8d70ae479 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.193126 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fe5c4-c4af-42aa-b46e-3eb8d70ae479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.193396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fe5c4-c4af-42aa-b46e-3eb8d70ae479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.194081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fe5c4-c4af-42aa-b46e-3eb8d70ae479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.196344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fe5c4-c4af-42aa-b46e-3eb8d70ae479 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.212788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1143/2322] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:44.213638 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fe5c4-c4af-42aa-b46e-3eb8d70ae479 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.223168 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fe5c4-c4af-42aa-b46e-3eb8d70ae479 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.227807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba011d9a-de7c-4e9a-861d-3795498a65a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.228434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba011d9a-de7c-4e9a-861d-3795498a65a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.228712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba011d9a-de7c-4e9a-861d-3795498a65a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.228983 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba011d9a-de7c-4e9a-861d-3795498a65a7 admin admin] PATH_INFO: `/v3/projects/6c4a9b3460ba4801a71fde87c99b5a10/users/6f7b24315693452ea26c68e0ae809402/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.244004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba011d9a-de7c-4e9a-861d-3795498a65a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4a9b3460ba4801a71fde87c99b5a10, user_id=6f7b24315693452ea26c68e0ae809402, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.246721 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba011d9a-de7c-4e9a-861d-3795498a65a7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.294187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1180/2323] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:44 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:45:44.296341 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9855af-28ce-40ce-8c83-778ed9a6fafe admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.306521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9855af-28ce-40ce-8c83-778ed9a6fafe admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.311072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5c601-0477-4349-b47f-64840bd3a7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.311635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5c601-0477-4349-b47f-64840bd3a7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.311910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5c601-0477-4349-b47f-64840bd3a7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.312178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5c601-0477-4349-b47f-64840bd3a7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.312795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5c601-0477-4349-b47f-64840bd3a7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.314489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5c601-0477-4349-b47f-64840bd3a7c4 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.323706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1144/2324] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:44 2022] PUT /identity/v3/projects/6c4a9b3460ba4801a71fde87c99b5a10/users/6f7b24315693452ea26c68e0ae809402/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:44.327683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba011d9a-de7c-4e9a-861d-3795498a65a7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.329652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1181/2325] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:45:44.330657 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5c601-0477-4349-b47f-64840bd3a7c4 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.340183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5c601-0477-4349-b47f-64840bd3a7c4 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.392433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba011d9a-de7c-4e9a-861d-3795498a65a7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.397640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efc50a-7bad-41d4-b9f2-775b0a8b6b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.398202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efc50a-7bad-41d4-b9f2-775b0a8b6b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.398465 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efc50a-7bad-41d4-b9f2-775b0a8b6b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.398736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efc50a-7bad-41d4-b9f2-775b0a8b6b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.399370 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efc50a-7bad-41d4-b9f2-775b0a8b6b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.401043 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efc50a-7bad-41d4-b9f2-775b0a8b6b42 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.403280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c62aa-021a-43ea-86ed-554cee5c47fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.404498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c62aa-021a-43ea-86ed-554cee5c47fd admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.404729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c62aa-021a-43ea-86ed-554cee5c47fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.408649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c62aa-021a-43ea-86ed-554cee5c47fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.410033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c62aa-021a-43ea-86ed-554cee5c47fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.411317 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c62aa-021a-43ea-86ed-554cee5c47fd admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.413571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1145/2326] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:45:44.418024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efc50a-7bad-41d4-b9f2-775b0a8b6b42 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.426330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1182/2327] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:45:44.428542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efc50a-7bad-41d4-b9f2-775b0a8b6b42 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.429529 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c62aa-021a-43ea-86ed-554cee5c47fd admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.435480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757e74e-051e-415e-9261-81f1d4059273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.436360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757e74e-051e-415e-9261-81f1d4059273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.436745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757e74e-051e-415e-9261-81f1d4059273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.437026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757e74e-051e-415e-9261-81f1d4059273 admin admin] PATH_INFO: `/v3/projects/6c4a9b3460ba4801a71fde87c99b5a10/users/6f7b24315693452ea26c68e0ae809402/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.439027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757e74e-051e-415e-9261-81f1d4059273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4a9b3460ba4801a71fde87c99b5a10, user_id=6f7b24315693452ea26c68e0ae809402, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.441636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757e74e-051e-415e-9261-81f1d4059273 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.442609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c62aa-021a-43ea-86ed-554cee5c47fd admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.449041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfd598-1680-4e70-a276-81f430ce4db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.453217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd598-1680-4e70-a276-81f430ce4db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.453217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd598-1680-4e70-a276-81f430ce4db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.453217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd598-1680-4e70-a276-81f430ce4db5 admin admin] PATH_INFO: `/v3/projects/58faec21e63d41bf9a3ff1aec775aa81/users/622af4e192554d299301f9284eb9c2d7/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.453972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1146/2328] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:44 2022] PUT /identity/v3/projects/6c4a9b3460ba4801a71fde87c99b5a10/users/6f7b24315693452ea26c68e0ae809402/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:44.463262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777ff5f-e2b8-48e3-8f79-4e21df87103a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.463651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777ff5f-e2b8-48e3-8f79-4e21df87103a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.463989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777ff5f-e2b8-48e3-8f79-4e21df87103a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.473630 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfd598-1680-4e70-a276-81f430ce4db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58faec21e63d41bf9a3ff1aec775aa81, user_id=622af4e192554d299301f9284eb9c2d7, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.476424 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfd598-1680-4e70-a276-81f430ce4db5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.493649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6777ff5f-e2b8-48e3-8f79-4e21df87103a None None] MFA Rules not processed for user `6f7b24315693452ea26c68e0ae809402`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:44.494900 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6777ff5f-e2b8-48e3-8f79-4e21df87103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:44.544492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:44.562374 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1183/2329] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:44 2022] PUT /identity/v3/projects/58faec21e63d41bf9a3ff1aec775aa81/users/622af4e192554d299301f9284eb9c2d7/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:44.567939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70caed38-5523-4419-810a-2a6a0f034c28 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.568228 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70caed38-5523-4419-810a-2a6a0f034c28 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.568566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70caed38-5523-4419-810a-2a6a0f034c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.578235 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1147/2330] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:44.589757 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777ff5f-e2b8-48e3-8f79-4e21df87103a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.591286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6777ff5f-e2b8-48e3-8f79-4e21df87103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:44.602631 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-70caed38-5523-4419-810a-2a6a0f034c28 None None] MFA Rules not processed for user `622af4e192554d299301f9284eb9c2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:44.604012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-70caed38-5523-4419-810a-2a6a0f034c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:44.618512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:44.658520 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777ff5f-e2b8-48e3-8f79-4e21df87103a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.673382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1184/2331] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:44.685650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70caed38-5523-4419-810a-2a6a0f034c28 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.687061 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-70caed38-5523-4419-810a-2a6a0f034c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:44.724765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db05a-cf86-4c06-9ffb-8fc586934295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.725835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db05a-cf86-4c06-9ffb-8fc586934295 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.726305 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db05a-cf86-4c06-9ffb-8fc586934295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.727092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db05a-cf86-4c06-9ffb-8fc586934295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.728990 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3e7db05a-cf86-4c06-9ffb-8fc586934295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:44.755625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70caed38-5523-4419-810a-2a6a0f034c28 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.763277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31099c09-fc78-49c1-a587-84c235b70030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.763967 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099c09-fc78-49c1-a587-84c235b70030 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.764244 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099c09-fc78-49c1-a587-84c235b70030 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.764519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099c09-fc78-49c1-a587-84c235b70030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.776681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31099c09-fc78-49c1-a587-84c235b70030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.778517 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31099c09-fc78-49c1-a587-84c235b70030 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.788642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1185/2332] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:45:44.797229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7db05a-cf86-4c06-9ffb-8fc586934295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.798810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7db05a-cf86-4c06-9ffb-8fc586934295 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:44.813263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1148/2333] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:44.814448 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31099c09-fc78-49c1-a587-84c235b70030 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:44.879502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31099c09-fc78-49c1-a587-84c235b70030 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:44.884502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:44.885132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:44.885523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:44.885829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] PATH_INFO: `/v3/users/622af4e192554d299301f9284eb9c2d7` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:44.887021 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=622af4e192554d299301f9284eb9c2d7)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:44.888950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:45.101455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622af4e192554d299301f9284eb9c2d7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:45.119908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:45.122390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622af4e192554d299301f9284eb9c2d7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:45.123099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622af4e192554d299301f9284eb9c2d7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:45.213330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622af4e192554d299301f9284eb9c2d7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:45.220775 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622af4e192554d299301f9284eb9c2d7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:45.227350 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1186/2334] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:44 2022] DELETE /identity/v3/users/622af4e192554d299301f9284eb9c2d7 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:45.231985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16a148-747d-4904-9e43-314852099917 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:45.232429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16a148-747d-4904-9e43-314852099917 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:45.232713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16a148-747d-4904-9e43-314852099917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:45.252165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ce16a148-747d-4904-9e43-314852099917 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:45.253295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ce16a148-747d-4904-9e43-314852099917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:45.305259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:45.314654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1149/2335] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:45.324914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:45.326504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:45.387379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7c74c-162a-44e6-a4bb-c55242546153 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:45.390992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ba74f-a08a-4b8c-8bdb-e30bd98b0f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:45.391589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ba74f-a08a-4b8c-8bdb-e30bd98b0f2f service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:45.391867 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ba74f-a08a-4b8c-8bdb-e30bd98b0f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:45.392137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ba74f-a08a-4b8c-8bdb-e30bd98b0f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:45.401834 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ba74f-a08a-4b8c-8bdb-e30bd98b0f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:45.403286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ba74f-a08a-4b8c-8bdb-e30bd98b0f2f service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:45.412787 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1187/2336] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:45.828439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16a148-747d-4904-9e43-314852099917 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:45.830906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ce16a148-747d-4904-9e43-314852099917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:45.854711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16a148-747d-4904-9e43-314852099917 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:45.859676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:45.860328 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:45.860593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:45.860868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] PATH_INFO: `/v3/projects/58faec21e63d41bf9a3ff1aec775aa81` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:45.862031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58faec21e63d41bf9a3ff1aec775aa81)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:45.864106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:45.979118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58faec21e63d41bf9a3ff1aec775aa81', 'request_id': 'req-31a7bcda-9779-436c-afbe-bb183d9654d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:45.985075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58faec21e63d41bf9a3ff1aec775aa81', 'request_id': 'req-31a7bcda-9779-436c-afbe-bb183d9654d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:45.991752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58faec21e63d41bf9a3ff1aec775aa81', 'request_id': 'req-31a7bcda-9779-436c-afbe-bb183d9654d1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:46.076292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] The token cache is being invalidate because project 58faec21e63d41bf9a3ff1aec775aa81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:46.076578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:46.077202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-31a7bcda-9779-436c-afbe-bb183d9654d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:46.078856 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1150/2337] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:45 2022] DELETE /identity/v3/projects/58faec21e63d41bf9a3ff1aec775aa81 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:46.084057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2229-6eb8-4ae1-a586-cf8000e42b57 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.084336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2229-6eb8-4ae1-a586-cf8000e42b57 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.084624 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb2229-6eb8-4ae1-a586-cf8000e42b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.104820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5efb2229-6eb8-4ae1-a586-cf8000e42b57 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:46.106149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5efb2229-6eb8-4ae1-a586-cf8000e42b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:46.164526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1188/2338] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:45:46.173704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f001-6c6a-448d-af82-bdbc55fefada None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.174018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f001-6c6a-448d-af82-bdbc55fefada None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.174285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f001-6c6a-448d-af82-bdbc55fefada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.194433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8332f001-6c6a-448d-af82-bdbc55fefada None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:46.195620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8332f001-6c6a-448d-af82-bdbc55fefada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:46.203450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1151/2339] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:46.207620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb2229-6eb8-4ae1-a586-cf8000e42b57 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:46.217420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb2229-6eb8-4ae1-a586-cf8000e42b57 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:46.222012 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:46.222736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.222957 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.223227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.223861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:46.224207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:46.225852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:46.241637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1189/2340] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:45:46.242962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332f001-6c6a-448d-af82-bdbc55fefada None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:46.253165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332f001-6c6a-448d-af82-bdbc55fefada None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:46.257949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eee130-2314-4131-ac03-04c771cb8aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:46.261570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee130-2314-4131-ac03-04c771cb8aae admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.261570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee130-2314-4131-ac03-04c771cb8aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.261570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eee130-2314-4131-ac03-04c771cb8aae admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.261570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eee130-2314-4131-ac03-04c771cb8aae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:46.261907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eee130-2314-4131-ac03-04c771cb8aae admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:46.380045 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1152/2341] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:46 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:46.384271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:46.394905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a18ba-67c8-43fa-b118-a88f514f3157 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:46.450844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bd9c6-ffc9-4631-902f-f5b8866cdbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:46.451450 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd9c6-ffc9-4631-902f-f5b8866cdbe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.451707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd9c6-ffc9-4631-902f-f5b8866cdbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.451980 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bd9c6-ffc9-4631-902f-f5b8866cdbe1 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.452605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bd9c6-ffc9-4631-902f-f5b8866cdbe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:46.454521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bd9c6-ffc9-4631-902f-f5b8866cdbe1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:46.527229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:46.533174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1190/2342] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:46 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:46.537516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eee130-2314-4131-ac03-04c771cb8aae admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:46.548643 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eee130-2314-4131-ac03-04c771cb8aae admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:46.553336 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601818b-40e7-4d4b-972f-cc69c4151a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:46.554021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601818b-40e7-4d4b-972f-cc69c4151a83 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.554314 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601818b-40e7-4d4b-972f-cc69c4151a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.554588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601818b-40e7-4d4b-972f-cc69c4151a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.555238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601818b-40e7-4d4b-972f-cc69c4151a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:46.556822 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601818b-40e7-4d4b-972f-cc69c4151a83 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:46.566959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1153/2343] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:46.570874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bd9c6-ffc9-4631-902f-f5b8866cdbe1 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:46.580804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bd9c6-ffc9-4631-902f-f5b8866cdbe1 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:46.585648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9a519-9bf1-4fba-89aa-7b4efced6367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:46.586276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9a519-9bf1-4fba-89aa-7b4efced6367 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.586558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9a519-9bf1-4fba-89aa-7b4efced6367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.587320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9a519-9bf1-4fba-89aa-7b4efced6367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.587959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9a519-9bf1-4fba-89aa-7b4efced6367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:46.589539 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9a519-9bf1-4fba-89aa-7b4efced6367 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:46.600525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1191/2344] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:46.605980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601818b-40e7-4d4b-972f-cc69c4151a83 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:46.616049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601818b-40e7-4d4b-972f-cc69c4151a83 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:46.621180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:46.621858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.622191 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.622525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] PATH_INFO: `/v3/projects/60e27bed3442483993b9252fd2ba1201/users/1808be9c1e8e4e908e87f286d15d128b/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.643874 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e27bed3442483993b9252fd2ba1201, user_id=1808be9c1e8e4e908e87f286d15d128b, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:46.646786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:46.732814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1154/2345] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:46 2022] PUT /identity/v3/projects/60e27bed3442483993b9252fd2ba1201/users/1808be9c1e8e4e908e87f286d15d128b/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:46.743226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bc08a-d5b2-4b8d-af17-e22a7c425cae None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.743508 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bc08a-d5b2-4b8d-af17-e22a7c425cae None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.743805 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bc08a-d5b2-4b8d-af17-e22a7c425cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.771960 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-1a7bc08a-d5b2-4b8d-af17-e22a7c425cae None None] MFA Rules not processed for user `1808be9c1e8e4e908e87f286d15d128b`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:46.773224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1a7bc08a-d5b2-4b8d-af17-e22a7c425cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:46.840122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1192/2346] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:46.850880 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:46.852673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:46.910404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b88c0b-b214-447b-91f5-82ac1c8f99ea admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:46.914056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:46.914677 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:46.914957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:46.915231 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:46.923486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:46.924941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:46.933767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1155/2347] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:47.109908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bc08a-d5b2-4b8d-af17-e22a7c425cae None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:47.111485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1a7bc08a-d5b2-4b8d-af17-e22a7c425cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:47.170305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bc08a-d5b2-4b8d-af17-e22a7c425cae None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:47.174123 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:47.174736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:47.175151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:47.175432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:47.177216 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:47.240383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:47.241792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:47.252604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1193/2348] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:49.017072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:49.019330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:49.034601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccbd0-13b6-40bc-a3f0-2529b82218f8 service cinder] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:49.040222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:49.042067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:49.042930 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:49.043323 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:49.045471 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:49.078293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:49.156202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:49.158081 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:49.182300 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1156/2349] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:50.216464 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:50.217770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:50.240546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366db98-7e43-49fc-b3e6-d307f6645e74 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:50.246614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:50.247562 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:50.248565 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:50.249113 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:50.251913 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:50.328115 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=100 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:50.424495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:50.426029 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:50.457423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1194/2350] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:50.970029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:50.971780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:51.005836 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcee5e-b7fa-4b64-9039-da250bfb3702 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:51.012366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:51.017330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:51.017330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:51.017330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] PATH_INFO: `/v3/users/df8da1fe608d4dd6acb6b7298ca56d45` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:51.017330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8da1fe608d4dd6acb6b7298ca56d45)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:51.018019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:51.335259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8da1fe608d4dd6acb6b7298ca56d45'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:51.348275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8da1fe608d4dd6acb6b7298ca56d45'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:51.446994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8da1fe608d4dd6acb6b7298ca56d45'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:51.447622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8da1fe608d4dd6acb6b7298ca56d45'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:51.455294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7dd582bb-528a-41c9-80e3-b4513039049d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8da1fe608d4dd6acb6b7298ca56d45'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:51.461364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1157/2351] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:45:50 2022] DELETE /identity/v3/users/df8da1fe608d4dd6acb6b7298ca56d45 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:51.775197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:51.785667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b599b-d029-44f0-8f4f-7126dd57c6ba service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:51.881485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:51.882290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:51.882635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:51.883042 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] PATH_INFO: `/v3/projects/c056ad05ef9843688f08b7e9f25e1138` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:51.884300 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c056ad05ef9843688f08b7e9f25e1138)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:51.886406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:51.989444 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c056ad05ef9843688f08b7e9f25e1138', 'request_id': 'req-e5559265-7c27-421b-b3fb-27c7df68e174'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:51.995498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c056ad05ef9843688f08b7e9f25e1138', 'request_id': 'req-e5559265-7c27-421b-b3fb-27c7df68e174'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:52.002386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c056ad05ef9843688f08b7e9f25e1138', 'request_id': 'req-e5559265-7c27-421b-b3fb-27c7df68e174'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:52.109589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] The token cache is being invalidate because project c056ad05ef9843688f08b7e9f25e1138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:45:52.109872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:52.110533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e5559265-7c27-421b-b3fb-27c7df68e174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:45:52.112010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1195/2352] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:45:51 2022] DELETE /identity/v3/projects/c056ad05ef9843688f08b7e9f25e1138 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:45:52.117071 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687371-fc3c-412d-8241-3b84889e3dd1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.117535 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687371-fc3c-412d-8241-3b84889e3dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.117641 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687371-fc3c-412d-8241-3b84889e3dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.139004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a8687371-fc3c-412d-8241-3b84889e3dd1 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:52.140234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a8687371-fc3c-412d-8241-3b84889e3dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:52.197748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1158/2353] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:52.205686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03516ff5-0bd2-4d4d-af3f-442828bebf20 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.206009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03516ff5-0bd2-4d4d-af3f-442828bebf20 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.206285 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03516ff5-0bd2-4d4d-af3f-442828bebf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.226283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-03516ff5-0bd2-4d4d-af3f-442828bebf20 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:52.227483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-03516ff5-0bd2-4d4d-af3f-442828bebf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:52.235429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1196/2354] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:52.239723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8687371-fc3c-412d-8241-3b84889e3dd1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:52.249630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8687371-fc3c-412d-8241-3b84889e3dd1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:52.255113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:52.255707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.256013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.256281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.256999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:45:52.257317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:52.259419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:52.275171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1159/2355] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:45:52.276231 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03516ff5-0bd2-4d4d-af3f-442828bebf20 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:52.285419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03516ff5-0bd2-4d4d-af3f-442828bebf20 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:52.290209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426204b-9170-43bd-b8fb-7c1c76916329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:52.290807 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426204b-9170-43bd-b8fb-7c1c76916329 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.291085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426204b-9170-43bd-b8fb-7c1c76916329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.291354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426204b-9170-43bd-b8fb-7c1c76916329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.291990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426204b-9170-43bd-b8fb-7c1c76916329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:52.293911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426204b-9170-43bd-b8fb-7c1c76916329 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:52.409108 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1197/2356] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:45:52 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:52.413222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:52.423042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb587025-9abc-4e0f-b29a-51aacce3988f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:52.435753 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:52.479791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eee9f-a798-481c-9649-efe311ab4e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:52.480371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eee9f-a798-481c-9649-efe311ab4e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.480646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eee9f-a798-481c-9649-efe311ab4e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.480915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eee9f-a798-481c-9649-efe311ab4e28 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.481541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9eee9f-a798-481c-9649-efe311ab4e28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:52.483586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9eee9f-a798-481c-9649-efe311ab4e28 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:52.580079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1160/2357] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:45:52 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:45:52.584065 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426204b-9170-43bd-b8fb-7c1c76916329 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:52.594682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426204b-9170-43bd-b8fb-7c1c76916329 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:52.599503 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0adba-3635-4d15-a3b0-7ea6356cea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:52.600102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0adba-3635-4d15-a3b0-7ea6356cea81 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.600381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0adba-3635-4d15-a3b0-7ea6356cea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.600655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0adba-3635-4d15-a3b0-7ea6356cea81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.601344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0adba-3635-4d15-a3b0-7ea6356cea81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:52.602991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0adba-3635-4d15-a3b0-7ea6356cea81 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:52.613164 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1198/2358] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:52.617092 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eee9f-a798-481c-9649-efe311ab4e28 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:52.628126 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eee9f-a798-481c-9649-efe311ab4e28 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:52.632725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84126b-d46b-4591-bebc-c301460bc041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:52.633309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84126b-d46b-4591-bebc-c301460bc041 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.633584 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84126b-d46b-4591-bebc-c301460bc041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.634512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84126b-d46b-4591-bebc-c301460bc041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.635150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84126b-d46b-4591-bebc-c301460bc041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:52.636723 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84126b-d46b-4591-bebc-c301460bc041 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:52.647330 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1161/2359] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:45:52.651414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0adba-3635-4d15-a3b0-7ea6356cea81 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:52.661344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0adba-3635-4d15-a3b0-7ea6356cea81 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:52.665844 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:52.666502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.666778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.667048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] PATH_INFO: `/v3/projects/3ebab900aef740b68af799c4695e6e5f/users/f84e6adb1a0e4d709b2962c9793346dc/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.675323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:52.682068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ebab900aef740b68af799c4695e6e5f, user_id=f84e6adb1a0e4d709b2962c9793346dc, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:52.684750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:52.793443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1199/2360] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:45:52 2022] PUT /identity/v3/projects/3ebab900aef740b68af799c4695e6e5f/users/f84e6adb1a0e4d709b2962c9793346dc/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:45:52.796270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43642ff-11e5-46a6-837b-eb446db24a4e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.796569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43642ff-11e5-46a6-837b-eb446db24a4e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.796829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43642ff-11e5-46a6-837b-eb446db24a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.816105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-e43642ff-11e5-46a6-837b-eb446db24a4e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:52.817296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e43642ff-11e5-46a6-837b-eb446db24a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:52.875942 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1162/2361] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:52.885454 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:52.887170 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:52.943588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f305b-bdd8-45fb-badc-fda96841e186 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:52.947252 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:52.947840 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:52.948119 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:52.948389 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:52.957150 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:52.958593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:52.967883 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1200/2362] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:58.824745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636176-1a7d-4a12-92a0-438bc35ea429 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:58.825237 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636176-1a7d-4a12-92a0-438bc35ea429 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:58.825339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636176-1a7d-4a12-92a0-438bc35ea429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:58.860597 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-33636176-1a7d-4a12-92a0-438bc35ea429 None None] MFA Rules not processed for user `f84e6adb1a0e4d709b2962c9793346dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:45:58.861842 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-33636176-1a7d-4a12-92a0-438bc35ea429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:58.958170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1163/2363] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:45:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:58.979158 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:58.980597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:59.065781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664e1b3-a456-4262-9df6-acd4412801f5 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:59.077804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dde4a7-5019-4558-8e54-3d449504b43a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:59.078468 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde4a7-5019-4558-8e54-3d449504b43a service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:59.078748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde4a7-5019-4558-8e54-3d449504b43a service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:59.079023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde4a7-5019-4558-8e54-3d449504b43a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:59.088997 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dde4a7-5019-4558-8e54-3d449504b43a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:59.090632 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dde4a7-5019-4558-8e54-3d449504b43a service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:59.101322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1201/2364] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:59.766516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33636176-1a7d-4a12-92a0-438bc35ea429 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:59.776979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33636176-1a7d-4a12-92a0-438bc35ea429 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:59.781129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:59.781775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:59.782086 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:59.782361 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:59.784102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:59.798149 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:59.799534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:59.811278 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1164/2365] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:01.935747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c0fe-7aee-42e3-9b21-b14df59c74ba None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:01.936340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c0fe-7aee-42e3-9b21-b14df59c74ba None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:01.936589 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c0fe-7aee-42e3-9b21-b14df59c74ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:01.967427 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ba54c0fe-7aee-42e3-9b21-b14df59c74ba None None] MFA Rules not processed for user `f84e6adb1a0e4d709b2962c9793346dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:01.969041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ba54c0fe-7aee-42e3-9b21-b14df59c74ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:01.978854 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1202/2366] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:04.860296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:04.861765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:04.877790 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:04.946617 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476475-598b-4fd6-9e9f-475ee0cfe9bd service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:04.951423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:04.952064 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:04.952345 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:04.952618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] PATH_INFO: `/v3/users/1808be9c1e8e4e908e87f286d15d128b` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:04.953756 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1808be9c1e8e4e908e87f286d15d128b)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:04.955800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:05.236464 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1808be9c1e8e4e908e87f286d15d128b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:05.248929 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1808be9c1e8e4e908e87f286d15d128b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:05.310469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1808be9c1e8e4e908e87f286d15d128b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:05.311029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1808be9c1e8e4e908e87f286d15d128b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:05.319804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1808be9c1e8e4e908e87f286d15d128b'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:05.325858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1165/2367] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:04 2022] DELETE /identity/v3/users/1808be9c1e8e4e908e87f286d15d128b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:05.330576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a5857-7efc-4dd6-a63a-7c67059d84b7 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:05.331041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a5857-7efc-4dd6-a63a-7c67059d84b7 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:05.331125 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a5857-7efc-4dd6-a63a-7c67059d84b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:05.360354 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-ea7a5857-7efc-4dd6-a63a-7c67059d84b7 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:05.361560 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ea7a5857-7efc-4dd6-a63a-7c67059d84b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:05.420971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1203/2368] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:05.431526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:05.433079 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:05.516406 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edaf8d-eca5-4204-8b46-4855e6cb56a4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:05.520201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:05.520818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:05.521141 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:05.521513 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:05.532785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:05.537556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:05.546325 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1166/2369] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:05.981067 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a5857-7efc-4dd6-a63a-7c67059d84b7 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:05.985152 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ea7a5857-7efc-4dd6-a63a-7c67059d84b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:06.007966 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:06.014636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a5857-7efc-4dd6-a63a-7c67059d84b7 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:06.023781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:06.024686 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:06.028462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:06.028462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] PATH_INFO: `/v3/projects/60e27bed3442483993b9252fd2ba1201` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:06.028462 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e27bed3442483993b9252fd2ba1201)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:06.028756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:06.141059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e27bed3442483993b9252fd2ba1201', 'request_id': 'req-d8dd393c-2e33-4e22-9684-db1dc8062ede'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:06.148126 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e27bed3442483993b9252fd2ba1201', 'request_id': 'req-d8dd393c-2e33-4e22-9684-db1dc8062ede'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:06.155617 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e27bed3442483993b9252fd2ba1201', 'request_id': 'req-d8dd393c-2e33-4e22-9684-db1dc8062ede'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:06.264309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] The token cache is being invalidate because project 60e27bed3442483993b9252fd2ba1201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:06.264601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:06.265431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:06.267384 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1204/2370] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:05 2022] DELETE /identity/v3/projects/60e27bed3442483993b9252fd2ba1201 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:08.504555 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:08.506243 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:08.533265 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c4944-e486-4ea3-8a18-bb7e09f93145 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:08.542266 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:08.612387 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:08.613032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:08.615051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:08.615051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] PATH_INFO: `/v3/users/b0a7f349d8ba4bea9a609e7cf881b1b9` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:08.615051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a7f349d8ba4bea9a609e7cf881b1b9)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:08.616716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:08.800299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a7f349d8ba4bea9a609e7cf881b1b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:08.820226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a7f349d8ba4bea9a609e7cf881b1b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:08.865085 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a7f349d8ba4bea9a609e7cf881b1b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:08.865830 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a7f349d8ba4bea9a609e7cf881b1b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:08.877630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a7f349d8ba4bea9a609e7cf881b1b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:08.884091 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1167/2371] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:08 2022] DELETE /identity/v3/users/b0a7f349d8ba4bea9a609e7cf881b1b9 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:09.370584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:09.372120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:09.386592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd393c-2e33-4e22-9684-db1dc8062ede admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:09.450998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:09.451629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:09.451911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:09.452184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] PATH_INFO: `/v3/projects/b15d78e196664c1aa41be6c5afe4238b` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:09.453333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b15d78e196664c1aa41be6c5afe4238b)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:09.455283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:09.591273 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b15d78e196664c1aa41be6c5afe4238b', 'request_id': 'req-f3dd9571-5422-408b-b6ae-71f5c44b6dba'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:09.596804 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b15d78e196664c1aa41be6c5afe4238b', 'request_id': 'req-f3dd9571-5422-408b-b6ae-71f5c44b6dba'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:09.603784 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b15d78e196664c1aa41be6c5afe4238b', 'request_id': 'req-f3dd9571-5422-408b-b6ae-71f5c44b6dba'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:09.699489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] The token cache is being invalidate because project b15d78e196664c1aa41be6c5afe4238b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:09.709707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:09.710947 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:09.713040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1205/2372] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:09 2022] DELETE /identity/v3/projects/b15d78e196664c1aa41be6c5afe4238b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:09.717000 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:09.718444 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:09.734798 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b19115-a368-4d5d-8f02-4bbd6de5fbc8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:09.794803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:09.795431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:09.795704 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:09.796497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] PATH_INFO: `/v3/users/be7509d5cdca46be91f7259e320efd5d` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:09.797105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] RBAC: Authorizing `identity:delete_user(user_id=be7509d5cdca46be91f7259e320efd5d)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:09.799055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:09.964570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:09.976982 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be7509d5cdca46be91f7259e320efd5d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:09.990561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be7509d5cdca46be91f7259e320efd5d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.110979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be7509d5cdca46be91f7259e320efd5d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.111776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be7509d5cdca46be91f7259e320efd5d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.122888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be7509d5cdca46be91f7259e320efd5d'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.154110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1168/2373] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:09 2022] DELETE /identity/v3/users/be7509d5cdca46be91f7259e320efd5d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:10.413845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:10.416110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:10.445180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd9571-5422-408b-b6ae-71f5c44b6dba admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:10.458383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:10.504296 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:10.504921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:10.505198 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:10.505466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] PATH_INFO: `/v3/projects/26d5ba2a9a0b401ea29a47f5b0e047e6` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:10.506713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d5ba2a9a0b401ea29a47f5b0e047e6)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:10.508667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:10.607323 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d5ba2a9a0b401ea29a47f5b0e047e6', 'request_id': 'req-04369df1-2183-48d6-bf31-ce317e4fa027'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.612699 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d5ba2a9a0b401ea29a47f5b0e047e6', 'request_id': 'req-04369df1-2183-48d6-bf31-ce317e4fa027'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.619755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d5ba2a9a0b401ea29a47f5b0e047e6', 'request_id': 'req-04369df1-2183-48d6-bf31-ce317e4fa027'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.704001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] The token cache is being invalidate because project 26d5ba2a9a0b401ea29a47f5b0e047e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:10.704290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.704927 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:10.706604 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1206/2374] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:10 2022] DELETE /identity/v3/projects/26d5ba2a9a0b401ea29a47f5b0e047e6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:15.340482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:15.342093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:15.439968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad659c3-c73b-4465-8334-c2ab26ee755e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:15.498882 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c019ba2-ad33-41d3-9955-80c79d9dd277 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:15.499489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c019ba2-ad33-41d3-9955-80c79d9dd277 service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:15.499769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c019ba2-ad33-41d3-9955-80c79d9dd277 service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:15.500041 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c019ba2-ad33-41d3-9955-80c79d9dd277 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:15.502042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3c019ba2-ad33-41d3-9955-80c79d9dd277 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:15.512948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c019ba2-ad33-41d3-9955-80c79d9dd277 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:15.514550 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c019ba2-ad33-41d3-9955-80c79d9dd277 service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:15.524393 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1169/2375] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:18.804620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:18.806193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:18.872611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04369df1-2183-48d6-bf31-ce317e4fa027 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:18.877511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:18.878171 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:18.878449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:18.878717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] PATH_INFO: `/v3/users/e0ed51c8873c47c388da4968ccf99ee0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:18.879815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ed51c8873c47c388da4968ccf99ee0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:18.881711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:19.102118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ed51c8873c47c388da4968ccf99ee0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:19.103351 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:19.114601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ed51c8873c47c388da4968ccf99ee0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:19.115279 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ed51c8873c47c388da4968ccf99ee0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:19.190578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ed51c8873c47c388da4968ccf99ee0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:19.198149 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ed51c8873c47c388da4968ccf99ee0'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:19.203950 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1207/2376] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:18 2022] DELETE /identity/v3/users/e0ed51c8873c47c388da4968ccf99ee0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:19.208482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb90d1-6efd-48bd-b676-0b1e89fe7c87 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:19.208960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb90d1-6efd-48bd-b676-0b1e89fe7c87 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:19.209067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb90d1-6efd-48bd-b676-0b1e89fe7c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:19.228784 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-22fb90d1-6efd-48bd-b676-0b1e89fe7c87 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:19.230008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22fb90d1-6efd-48bd-b676-0b1e89fe7c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:19.246216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:19.290056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1170/2377] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:19.300175 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:19.301665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:19.364499 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c439aa1-722c-4db4-af38-2c97485812b9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:19.369578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e8130-6274-4645-b0ca-1f3ed5760303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:19.370284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e8130-6274-4645-b0ca-1f3ed5760303 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:19.370564 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e8130-6274-4645-b0ca-1f3ed5760303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:19.370832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e8130-6274-4645-b0ca-1f3ed5760303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:19.380661 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e8130-6274-4645-b0ca-1f3ed5760303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:19.382109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e8130-6274-4645-b0ca-1f3ed5760303 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:19.392584 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1208/2378] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:46:19.861659 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb90d1-6efd-48bd-b676-0b1e89fe7c87 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:19.863339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22fb90d1-6efd-48bd-b676-0b1e89fe7c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:19.885989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb90d1-6efd-48bd-b676-0b1e89fe7c87 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:19.892563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:19.893223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:19.893507 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:19.893785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] PATH_INFO: `/v3/projects/dbd890d552c442dcb9aa00a2fca42818` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:19.894975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd890d552c442dcb9aa00a2fca42818)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:19.897044 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:20.039007 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd890d552c442dcb9aa00a2fca42818', 'request_id': 'req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.046959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd890d552c442dcb9aa00a2fca42818', 'request_id': 'req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.053816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd890d552c442dcb9aa00a2fca42818', 'request_id': 'req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.202306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] The token cache is being invalidate because project dbd890d552c442dcb9aa00a2fca42818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:20.202595 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.203503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.205313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1171/2379] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:19 2022] DELETE /identity/v3/projects/dbd890d552c442dcb9aa00a2fca42818 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:20.209404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e8130-6274-4645-b0ca-1f3ed5760303 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:20.219676 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e8130-6274-4645-b0ca-1f3ed5760303 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:20.278620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:20.279182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:20.279552 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:20.279835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] PATH_INFO: `/v3/users/6f7b24315693452ea26c68e0ae809402` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:20.281040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7b24315693452ea26c68e0ae809402)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:20.283081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:20.519174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7b24315693452ea26c68e0ae809402'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.532440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7b24315693452ea26c68e0ae809402'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.533061 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7b24315693452ea26c68e0ae809402'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.536739 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:20.606550 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7b24315693452ea26c68e0ae809402'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.615162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-c08aad6f-f5b2-40bb-9911-65cf02fa5af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7b24315693452ea26c68e0ae809402'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:20.621263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1209/2380] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:20 2022] DELETE /identity/v3/users/6f7b24315693452ea26c68e0ae809402 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:21.169927 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:21.170884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:21.184995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93c05e-02e7-44c2-9f15-e0f816b4ceb7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:21.255482 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:21.256312 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:21.256691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:21.257065 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] PATH_INFO: `/v3/projects/6c4a9b3460ba4801a71fde87c99b5a10` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:21.258512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c4a9b3460ba4801a71fde87c99b5a10)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:21.260578 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:21.378462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:21.380195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c4a9b3460ba4801a71fde87c99b5a10', 'request_id': 'req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:21.388122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c4a9b3460ba4801a71fde87c99b5a10', 'request_id': 'req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:21.395261 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c4a9b3460ba4801a71fde87c99b5a10', 'request_id': 'req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:21.522625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] The token cache is being invalidate because project 6c4a9b3460ba4801a71fde87c99b5a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:21.523160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:21.523898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-d1bab036-6ad9-4b9e-ab80-f9aa05422fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:21.525837 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1172/2381] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:21 2022] DELETE /identity/v3/projects/6c4a9b3460ba4801a71fde87c99b5a10 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:21.530984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3b18-1e37-43a5-b9bd-a3afe49bc59c None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:21.531548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3b18-1e37-43a5-b9bd-a3afe49bc59c None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:21.531548 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd3b18-1e37-43a5-b9bd-a3afe49bc59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:21.584368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-b3fd3b18-1e37-43a5-b9bd-a3afe49bc59c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:21.585598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-b3fd3b18-1e37-43a5-b9bd-a3afe49bc59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:21.652455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1210/2382] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:21.660519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356894-dc88-42f8-936e-f073ec0b605c None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:21.660850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356894-dc88-42f8-936e-f073ec0b605c None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:21.661110 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356894-dc88-42f8-936e-f073ec0b605c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:21.682171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3c356894-dc88-42f8-936e-f073ec0b605c None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:21.683391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3c356894-dc88-42f8-936e-f073ec0b605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:21.691605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1173/2383] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:21.696037 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd3b18-1e37-43a5-b9bd-a3afe49bc59c None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:21.706795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd3b18-1e37-43a5-b9bd-a3afe49bc59c None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:21.711471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:21.712075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:21.715404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:21.715404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:21.715404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:46:21.715404 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:21.715835 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:21.728092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1211/2384] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:46:21.733274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c356894-dc88-42f8-936e-f073ec0b605c None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:21.742878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c356894-dc88-42f8-936e-f073ec0b605c None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:21.748169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2935f-8e75-44a1-926c-b3fbe66b5f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:21.749857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2935f-8e75-44a1-926c-b3fbe66b5f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:21.749857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2935f-8e75-44a1-926c-b3fbe66b5f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:21.749857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2935f-8e75-44a1-926c-b3fbe66b5f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:21.750279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2935f-8e75-44a1-926c-b3fbe66b5f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:21.752120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2935f-8e75-44a1-926c-b3fbe66b5f54 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:21.849453 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1174/2385] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:46:21 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:46:21.856860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:21.868665 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2d379-844f-4f2a-a230-f17761039cc7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:21.930343 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8026e8-29c6-4edd-ac30-494ae792d4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:21.930958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8026e8-29c6-4edd-ac30-494ae792d4ed admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:21.931280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8026e8-29c6-4edd-ac30-494ae792d4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:21.931556 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8026e8-29c6-4edd-ac30-494ae792d4ed admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:21.932193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8026e8-29c6-4edd-ac30-494ae792d4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:21.934177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8026e8-29c6-4edd-ac30-494ae792d4ed admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.033185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1212/2386] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:46:21 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:46:22.037737 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2935f-8e75-44a1-926c-b3fbe66b5f54 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.049342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2935f-8e75-44a1-926c-b3fbe66b5f54 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.054344 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546a698-331d-4393-8f51-f502837d051f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.055026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546a698-331d-4393-8f51-f502837d051f admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.055371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546a698-331d-4393-8f51-f502837d051f admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.055706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546a698-331d-4393-8f51-f502837d051f admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.056418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546a698-331d-4393-8f51-f502837d051f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.058115 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546a698-331d-4393-8f51-f502837d051f admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.072706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1175/2387] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:46:22.073672 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8026e8-29c6-4edd-ac30-494ae792d4ed admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.083946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8026e8-29c6-4edd-ac30-494ae792d4ed admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.089709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15659b4-5833-44b0-b2d2-515062423dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.090337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15659b4-5833-44b0-b2d2-515062423dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.092658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15659b4-5833-44b0-b2d2-515062423dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.092658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15659b4-5833-44b0-b2d2-515062423dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.092658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15659b4-5833-44b0-b2d2-515062423dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.093226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15659b4-5833-44b0-b2d2-515062423dd1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.106501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1213/2388] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:46:22.107736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546a698-331d-4393-8f51-f502837d051f admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.117886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546a698-331d-4393-8f51-f502837d051f admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.122588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c85c8-1a95-42c2-b978-171559ae7fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.123181 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c85c8-1a95-42c2-b978-171559ae7fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.123480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c85c8-1a95-42c2-b978-171559ae7fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.123745 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c85c8-1a95-42c2-b978-171559ae7fb3 admin admin] PATH_INFO: `/v3/projects/6fb55aef36584f64ab8471da3a0eeb2b/users/8879c6c118cd4e89813933465feaefdb/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.140366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c85c8-1a95-42c2-b978-171559ae7fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb55aef36584f64ab8471da3a0eeb2b, user_id=8879c6c118cd4e89813933465feaefdb, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.143051 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c85c8-1a95-42c2-b978-171559ae7fb3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.242648 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1176/2389] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:46:22 2022] PUT /identity/v3/projects/6fb55aef36584f64ab8471da3a0eeb2b/users/8879c6c118cd4e89813933465feaefdb/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:22.253092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ccdc6-9fb6-479d-b9d3-378007b6047d None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.253402 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ccdc6-9fb6-479d-b9d3-378007b6047d None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.253679 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ccdc6-9fb6-479d-b9d3-378007b6047d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.283820 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-978ccdc6-9fb6-479d-b9d3-378007b6047d None None] MFA Rules not processed for user `8879c6c118cd4e89813933465feaefdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:22.285063 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-978ccdc6-9fb6-479d-b9d3-378007b6047d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:22.301707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:22.349308 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1214/2390] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:22.353869 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c85c8-1a95-42c2-b978-171559ae7fb3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.415541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c85c8-1a95-42c2-b978-171559ae7fb3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.420272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c6449-e2ef-4b38-ac3e-c7b32c3150d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.420867 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c6449-e2ef-4b38-ac3e-c7b32c3150d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.421147 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c6449-e2ef-4b38-ac3e-c7b32c3150d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.421421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c6449-e2ef-4b38-ac3e-c7b32c3150d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.422085 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c6449-e2ef-4b38-ac3e-c7b32c3150d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.423968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c6449-e2ef-4b38-ac3e-c7b32c3150d2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.517835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1177/2391] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:46:22 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:46:22.521885 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ccdc6-9fb6-479d-b9d3-378007b6047d None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.532557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ccdc6-9fb6-479d-b9d3-378007b6047d None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.589416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c6d3-87dd-45fa-8eaa-846ab7a61edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.590062 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c6d3-87dd-45fa-8eaa-846ab7a61edc admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.590344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c6d3-87dd-45fa-8eaa-846ab7a61edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.590622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14c6d3-87dd-45fa-8eaa-846ab7a61edc admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.591248 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14c6d3-87dd-45fa-8eaa-846ab7a61edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.593182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14c6d3-87dd-45fa-8eaa-846ab7a61edc admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.687666 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1215/2392] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:46:22 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:46:22.691609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c6449-e2ef-4b38-ac3e-c7b32c3150d2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.705301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c6449-e2ef-4b38-ac3e-c7b32c3150d2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.710069 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f01ce-5fbb-41cf-aa00-3e97016c86f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.710667 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f01ce-5fbb-41cf-aa00-3e97016c86f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.710945 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f01ce-5fbb-41cf-aa00-3e97016c86f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.711218 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f01ce-5fbb-41cf-aa00-3e97016c86f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.711854 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f01ce-5fbb-41cf-aa00-3e97016c86f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.713445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f01ce-5fbb-41cf-aa00-3e97016c86f9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.723992 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1178/2393] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:46:22.728642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c6d3-87dd-45fa-8eaa-846ab7a61edc admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.738650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14c6d3-87dd-45fa-8eaa-846ab7a61edc admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.743277 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46c4e6-9dee-4079-b46b-912f1cfa5e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.744651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c4e6-9dee-4079-b46b-912f1cfa5e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.744928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c4e6-9dee-4079-b46b-912f1cfa5e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.745204 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46c4e6-9dee-4079-b46b-912f1cfa5e00 admin admin] PATH_INFO: `/v3/projects/243555d709c64c85994ea4a097f229e9/users/85c8dcfe01b6491c9cc1841e8e9c48b9/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.759806 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46c4e6-9dee-4079-b46b-912f1cfa5e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243555d709c64c85994ea4a097f229e9, user_id=85c8dcfe01b6491c9cc1841e8e9c48b9, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.762491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46c4e6-9dee-4079-b46b-912f1cfa5e00 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.813623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1216/2394] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:46:22 2022] PUT /identity/v3/projects/243555d709c64c85994ea4a097f229e9/users/85c8dcfe01b6491c9cc1841e8e9c48b9/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:22.818215 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f01ce-5fbb-41cf-aa00-3e97016c86f9 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.828207 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f01ce-5fbb-41cf-aa00-3e97016c86f9 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.872873 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:22.885649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c68f5-0faf-4fec-ad64-513a450ce218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.886284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c68f5-0faf-4fec-ad64-513a450ce218 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.886561 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c68f5-0faf-4fec-ad64-513a450ce218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.886827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c68f5-0faf-4fec-ad64-513a450ce218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.887450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c68f5-0faf-4fec-ad64-513a450ce218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.889033 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c68f5-0faf-4fec-ad64-513a450ce218 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.898827 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1179/2395] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:46:22.903717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46c4e6-9dee-4079-b46b-912f1cfa5e00 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:22.914074 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46c4e6-9dee-4079-b46b-912f1cfa5e00 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:22.919167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:22.919808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.920087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.920357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] PATH_INFO: `/v3/projects/243555d709c64c85994ea4a097f229e9/users/85c8dcfe01b6491c9cc1841e8e9c48b9/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.922267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243555d709c64c85994ea4a097f229e9, user_id=85c8dcfe01b6491c9cc1841e8e9c48b9, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:22.924877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:22.935872 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1217/2396] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:46:22 2022] PUT /identity/v3/projects/243555d709c64c85994ea4a097f229e9/users/85c8dcfe01b6491c9cc1841e8e9c48b9/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:22.946324 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df125354-0b7f-49f2-8ce4-1ad271cd14c9 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:22.946554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df125354-0b7f-49f2-8ce4-1ad271cd14c9 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:22.946831 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df125354-0b7f-49f2-8ce4-1ad271cd14c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:22.975317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-df125354-0b7f-49f2-8ce4-1ad271cd14c9 None None] MFA Rules not processed for user `85c8dcfe01b6491c9cc1841e8e9c48b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:22.976518 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-df125354-0b7f-49f2-8ce4-1ad271cd14c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:23.046531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1180/2397] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:23.059411 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:23.061102 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:23.119793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0513-0936-4f92-be3e-66d6c3a64282 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:23.123432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f2cc-d4ea-40e3-9cd6-c0497bde5ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:23.124026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f2cc-d4ea-40e3-9cd6-c0497bde5ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:23.124302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f2cc-d4ea-40e3-9cd6-c0497bde5ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:23.124570 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f2cc-d4ea-40e3-9cd6-c0497bde5ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:23.133595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6f2cc-d4ea-40e3-9cd6-c0497bde5ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:23.135041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6f2cc-d4ea-40e3-9cd6-c0497bde5ff9 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:23.144283 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1218/2398] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:24.122885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df125354-0b7f-49f2-8ce4-1ad271cd14c9 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:24.132267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df125354-0b7f-49f2-8ce4-1ad271cd14c9 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:24.137434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85c8dcfe01b6491c9cc1841e8e9c48b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243555d709c64c85994ea4a097f229e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:24.138059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:24.138346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:24.138625 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:24.139271 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:46:24.139577 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:24.141199 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:24.152558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1181/2399] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:46:24.156609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f2cc-d4ea-40e3-9cd6-c0497bde5ff9 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:24.165627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f2cc-d4ea-40e3-9cd6-c0497bde5ff9 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:24.170322 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e867afb-2f51-4793-8289-fca0e792c66d tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85c8dcfe01b6491c9cc1841e8e9c48b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243555d709c64c85994ea4a097f229e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:24.170919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867afb-2f51-4793-8289-fca0e792c66d tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:24.171200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867afb-2f51-4793-8289-fca0e792c66d tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:24.171475 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867afb-2f51-4793-8289-fca0e792c66d tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:24.172108 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e867afb-2f51-4793-8289-fca0e792c66d tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:24.174040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e867afb-2f51-4793-8289-fca0e792c66d tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:24.185567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:24.264944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1219/2400] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:46:24 2022] POST /identity/v3/projects => generated 355 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:46:24.268655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:24.278685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21797d9d-6938-41bc-ac58-a2c3000fc8e4 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:24.335114 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85c8dcfe01b6491c9cc1841e8e9c48b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243555d709c64c85994ea4a097f229e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:24.335640 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:24.335921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:24.336197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:24.336858 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:46:24.337150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:24.338804 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:24.349926 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1182/2401] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:46:24.388588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e867afb-2f51-4793-8289-fca0e792c66d tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:24.398713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e867afb-2f51-4793-8289-fca0e792c66d tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:24.403370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85c8dcfe01b6491c9cc1841e8e9c48b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '243555d709c64c85994ea4a097f229e9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:24.403985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:24.404262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:24.405182 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] PATH_INFO: `/v3/projects/e6266aeea48e46a190f9d2d2d64a4843` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:24.405768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e6266aeea48e46a190f9d2d2d64a4843)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:24.407727 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:24.544571 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6266aeea48e46a190f9d2d2d64a4843', 'request_id': 'req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:24.550213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6266aeea48e46a190f9d2d2d64a4843', 'request_id': 'req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:24.557137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6266aeea48e46a190f9d2d2d64a4843', 'request_id': 'req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:24.628869 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] The token cache is being invalidate because project e6266aeea48e46a190f9d2d2d64a4843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:24.629087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:24.629714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-f1b1bd1e-b869-4c44-aafe-5d3832f9ef24 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:24.631239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1220/2402] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:24 2022] DELETE /identity/v3/projects/e6266aeea48e46a190f9d2d2d64a4843 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:25.137851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:25.139618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:25.189029 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=100 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:25.203828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a79dc-766a-4c42-bbc1-572d75336561 tempest-VolumeQuotaClassesTest-1544706004 tempest-VolumeQuotaClassesTest-1544706004-project-admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:25.259171 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:25.259800 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:25.260082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:25.260356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] PATH_INFO: `/v3/users/8879c6c118cd4e89813933465feaefdb` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:25.261484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8879c6c118cd4e89813933465feaefdb)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:25.263434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:25.404396 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8879c6c118cd4e89813933465feaefdb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:25.417154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8879c6c118cd4e89813933465feaefdb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:25.464872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8879c6c118cd4e89813933465feaefdb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:25.465441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8879c6c118cd4e89813933465feaefdb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:25.472692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8879c6c118cd4e89813933465feaefdb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:25.478559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1183/2403] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:25 2022] DELETE /identity/v3/users/8879c6c118cd4e89813933465feaefdb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:25.483106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72764b-5614-48f5-9191-6de56c9a0671 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:25.483588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72764b-5614-48f5-9191-6de56c9a0671 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:25.483682 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72764b-5614-48f5-9191-6de56c9a0671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:25.503107 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9c72764b-5614-48f5-9191-6de56c9a0671 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:25.504294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9c72764b-5614-48f5-9191-6de56c9a0671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:25.561506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1221/2404] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:25.571077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:25.573059 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:25.650460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c17e15-ba4f-4185-b469-f3af061204cf admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:25.654142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82722729-74f6-4d18-8018-31cce6ef3833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:25.654736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722729-74f6-4d18-8018-31cce6ef3833 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:25.655013 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722729-74f6-4d18-8018-31cce6ef3833 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:25.655281 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82722729-74f6-4d18-8018-31cce6ef3833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:25.664089 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82722729-74f6-4d18-8018-31cce6ef3833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:25.665559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82722729-74f6-4d18-8018-31cce6ef3833 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:25.669423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:25.675496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1184/2405] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:26.112577 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72764b-5614-48f5-9191-6de56c9a0671 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:26.114022 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9c72764b-5614-48f5-9191-6de56c9a0671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:26.126058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72764b-5614-48f5-9191-6de56c9a0671 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:26.131399 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:26.132195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:26.132636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:26.133013 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] PATH_INFO: `/v3/projects/6fb55aef36584f64ab8471da3a0eeb2b` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:26.134271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb55aef36584f64ab8471da3a0eeb2b)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:26.136263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:26.231800 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb55aef36584f64ab8471da3a0eeb2b', 'request_id': 'req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.237386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb55aef36584f64ab8471da3a0eeb2b', 'request_id': 'req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.243697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb55aef36584f64ab8471da3a0eeb2b', 'request_id': 'req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.283101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:26.287633 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] The token cache is being invalidate because project 6fb55aef36584f64ab8471da3a0eeb2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:26.287918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.288530 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.290030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1222/2406] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:26 2022] DELETE /identity/v3/projects/6fb55aef36584f64ab8471da3a0eeb2b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:26.294602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82722729-74f6-4d18-8018-31cce6ef3833 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:26.304364 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82722729-74f6-4d18-8018-31cce6ef3833 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:26.361421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:26.362003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:26.362287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:26.362558 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] PATH_INFO: `/v3/users/85c8dcfe01b6491c9cc1841e8e9c48b9` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:26.363698 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c8dcfe01b6491c9cc1841e8e9c48b9)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:26.365663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:26.558476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c8dcfe01b6491c9cc1841e8e9c48b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.571123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c8dcfe01b6491c9cc1841e8e9c48b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.646782 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c8dcfe01b6491c9cc1841e8e9c48b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.647418 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c8dcfe01b6491c9cc1841e8e9c48b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.655425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-671fbe6d-7ce5-4528-b628-5af80967ec53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c8dcfe01b6491c9cc1841e8e9c48b9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:26.661272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1185/2407] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:26 2022] DELETE /identity/v3/users/85c8dcfe01b6491c9cc1841e8e9c48b9 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:27.054201 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:27.056756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:27.071752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180960bf-c287-44fc-9bb9-b63d0ab4e6cc admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:27.130753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:27.131373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:27.131651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:27.131918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] PATH_INFO: `/v3/projects/243555d709c64c85994ea4a097f229e9` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:27.133008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=243555d709c64c85994ea4a097f229e9)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:27.135579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:27.237864 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243555d709c64c85994ea4a097f229e9', 'request_id': 'req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:27.244290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243555d709c64c85994ea4a097f229e9', 'request_id': 'req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:27.250625 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243555d709c64c85994ea4a097f229e9', 'request_id': 'req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:27.321595 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] The token cache is being invalidate because project 243555d709c64c85994ea4a097f229e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:27.321909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:27.322554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6a08da9e-6d9b-4e0a-b2c4-3d3ca8598bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:27.324051 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1223/2408] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:27 2022] DELETE /identity/v3/projects/243555d709c64c85994ea4a097f229e9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:27.328968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec114bde-5c86-411a-8b24-ca9f7ac92c0b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:27.329428 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec114bde-5c86-411a-8b24-ca9f7ac92c0b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:27.329542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec114bde-5c86-411a-8b24-ca9f7ac92c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:27.348964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-ec114bde-5c86-411a-8b24-ca9f7ac92c0b None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:27.350202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-ec114bde-5c86-411a-8b24-ca9f7ac92c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:27.408415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1186/2409] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:27.471428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec634-56db-416c-b5c7-5a764c197d33 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:27.471646 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec634-56db-416c-b5c7-5a764c197d33 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:27.471880 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec634-56db-416c-b5c7-5a764c197d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:27.491906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5a1ec634-56db-416c-b5c7-5a764c197d33 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:27.493105 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5a1ec634-56db-416c-b5c7-5a764c197d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:27.501795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1224/2410] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:27.505998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec114bde-5c86-411a-8b24-ca9f7ac92c0b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:27.515840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec114bde-5c86-411a-8b24-ca9f7ac92c0b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:27.520391 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:27.520986 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:27.521270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:27.521547 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:27.522224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:46:27.522533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:27.524124 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:27.535543 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1187/2411] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:46:27.539554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ec634-56db-416c-b5c7-5a764c197d33 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:27.549318 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ec634-56db-416c-b5c7-5a764c197d33 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:27.554829 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0fd91-a47c-4904-bfcb-9df1f9227c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:27.555423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fd91-a47c-4904-bfcb-9df1f9227c57 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:27.555703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fd91-a47c-4904-bfcb-9df1f9227c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:27.556000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0fd91-a47c-4904-bfcb-9df1f9227c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:27.557239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0fd91-a47c-4904-bfcb-9df1f9227c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:27.559177 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0fd91-a47c-4904-bfcb-9df1f9227c57 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:27.832663 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1225/2412] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:46:27 2022] POST /identity/v3/projects => generated 373 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:46:27.837042 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:27.843943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:27.847672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67586d1e-50b1-41cd-a5b4-caeecdefb471 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:27.903854 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42da5f3-534f-4962-9c79-68822ee295c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:27.904476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42da5f3-534f-4962-9c79-68822ee295c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:27.904752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42da5f3-534f-4962-9c79-68822ee295c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:27.905021 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42da5f3-534f-4962-9c79-68822ee295c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:27.905649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42da5f3-534f-4962-9c79-68822ee295c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:27.907725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42da5f3-534f-4962-9c79-68822ee295c5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:27.983910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1188/2413] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:46:27 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:46:27.988234 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0fd91-a47c-4904-bfcb-9df1f9227c57 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:27.998815 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0fd91-a47c-4904-bfcb-9df1f9227c57 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:28.003860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd800db-2fb2-46dc-b719-438d58aeb8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:28.004833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd800db-2fb2-46dc-b719-438d58aeb8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:28.004833 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd800db-2fb2-46dc-b719-438d58aeb8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:28.005124 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd800db-2fb2-46dc-b719-438d58aeb8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:28.005656 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd800db-2fb2-46dc-b719-438d58aeb8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:28.007302 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd800db-2fb2-46dc-b719-438d58aeb8c0 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:28.018147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1226/2414] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:46:28.021919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42da5f3-534f-4962-9c79-68822ee295c5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:28.032554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42da5f3-534f-4962-9c79-68822ee295c5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:28.037489 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed32eb2-1706-4b56-a77e-ffd62a0ae593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:28.038119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed32eb2-1706-4b56-a77e-ffd62a0ae593 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:28.038403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed32eb2-1706-4b56-a77e-ffd62a0ae593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:28.038674 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed32eb2-1706-4b56-a77e-ffd62a0ae593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:28.039311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed32eb2-1706-4b56-a77e-ffd62a0ae593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:28.040962 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed32eb2-1706-4b56-a77e-ffd62a0ae593 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:28.051289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1189/2415] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:46:28.055247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd800db-2fb2-46dc-b719-438d58aeb8c0 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:28.065281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd800db-2fb2-46dc-b719-438d58aeb8c0 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:28.069952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:28.070711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:28.070983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:28.071253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] PATH_INFO: `/v3/projects/f0f92c5af18446089c019fdedef0ffeb/users/98525c84eeb04cd4897d3e9804844e3c/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:28.086535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f92c5af18446089c019fdedef0ffeb, user_id=98525c84eeb04cd4897d3e9804844e3c, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:28.089770 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:28.191190 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1227/2416] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:46:28 2022] PUT /identity/v3/projects/f0f92c5af18446089c019fdedef0ffeb/users/98525c84eeb04cd4897d3e9804844e3c/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:28.201644 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7a4f4-cad5-4422-8eab-ab7816c625fc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:28.201953 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7a4f4-cad5-4422-8eab-ab7816c625fc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:28.202301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7a4f4-cad5-4422-8eab-ab7816c625fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:28.230525 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4ee7a4f4-cad5-4422-8eab-ab7816c625fc None None] MFA Rules not processed for user `98525c84eeb04cd4897d3e9804844e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:28.231740 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4ee7a4f4-cad5-4422-8eab-ab7816c625fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:28.296186 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1190/2417] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:28.307326 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:28.309000 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:28.336764 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:28.365395 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb66af-8519-464a-8cf9-cbcb4242880f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:28.369575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187812c-d0ee-4072-b6cb-84084283700f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:28.370203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187812c-d0ee-4072-b6cb-84084283700f service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:28.370482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187812c-d0ee-4072-b6cb-84084283700f service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:28.370751 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187812c-d0ee-4072-b6cb-84084283700f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:28.379523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187812c-d0ee-4072-b6cb-84084283700f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:28.380920 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187812c-d0ee-4072-b6cb-84084283700f service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:28.390386 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1228/2418] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:28.412208 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7a4f4-cad5-4422-8eab-ab7816c625fc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:28.473834 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7a4f4-cad5-4422-8eab-ab7816c625fc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:28.479087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:28.479701 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:28.479975 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:28.480247 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] PATH_INFO: `/v3/users/98525c84eeb04cd4897d3e9804844e3c` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:28.481363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98525c84eeb04cd4897d3e9804844e3c)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:28.483276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:28.496272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:28.668252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98525c84eeb04cd4897d3e9804844e3c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:28.680100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98525c84eeb04cd4897d3e9804844e3c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:28.744972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98525c84eeb04cd4897d3e9804844e3c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:28.745611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98525c84eeb04cd4897d3e9804844e3c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:28.753041 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98525c84eeb04cd4897d3e9804844e3c'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:28.759105 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1191/2419] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:28 2022] DELETE /identity/v3/users/98525c84eeb04cd4897d3e9804844e3c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:28.764072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0e47f-098f-42e1-85c8-444b55e356f3 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:28.764357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0e47f-098f-42e1-85c8-444b55e356f3 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:28.764636 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0e47f-098f-42e1-85c8-444b55e356f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:28.784355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-98f0e47f-098f-42e1-85c8-444b55e356f3 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:28.785546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-98f0e47f-098f-42e1-85c8-444b55e356f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:28.842650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1229/2420] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:28.852241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:28.853654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:28.910102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af841d-cd28-4b98-88a5-d45ec5a6ad41 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:28.913730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16847681-90fc-4f85-b604-eee5e634483a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:28.914354 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847681-90fc-4f85-b604-eee5e634483a service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:28.914633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847681-90fc-4f85-b604-eee5e634483a service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:28.914906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847681-90fc-4f85-b604-eee5e634483a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:28.923803 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16847681-90fc-4f85-b604-eee5e634483a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:28.925151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16847681-90fc-4f85-b604-eee5e634483a service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:28.934529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1192/2421] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:29.396621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0e47f-098f-42e1-85c8-444b55e356f3 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:29.398167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-98f0e47f-098f-42e1-85c8-444b55e356f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:29.411183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0e47f-098f-42e1-85c8-444b55e356f3 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:29.416546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:29.417263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:29.420696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:29.420696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] PATH_INFO: `/v3/projects/f0f92c5af18446089c019fdedef0ffeb` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:29.420696 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f92c5af18446089c019fdedef0ffeb)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:29.420996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:29.535200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f92c5af18446089c019fdedef0ffeb', 'request_id': 'req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:29.540959 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f92c5af18446089c019fdedef0ffeb', 'request_id': 'req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:29.547060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f92c5af18446089c019fdedef0ffeb', 'request_id': 'req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:29.629295 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] The token cache is being invalidate because project f0f92c5af18446089c019fdedef0ffeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:29.629579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:29.630232 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-de3b3721-7a06-4b8a-b3af-6acfbf1af5a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:29.631709 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1230/2422] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:29 2022] DELETE /identity/v3/projects/f0f92c5af18446089c019fdedef0ffeb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:29.638423 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828ae61-cd4b-4b1d-929d-4fe79c7a7df2 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:29.638911 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828ae61-cd4b-4b1d-929d-4fe79c7a7df2 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:29.639034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828ae61-cd4b-4b1d-929d-4fe79c7a7df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:29.658719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b828ae61-cd4b-4b1d-929d-4fe79c7a7df2 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:29.659910 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b828ae61-cd4b-4b1d-929d-4fe79c7a7df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:29.717378 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1193/2423] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:29.725085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1a0ea-c938-4f33-a437-76168a42a29f None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:29.725370 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1a0ea-c938-4f33-a437-76168a42a29f None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:29.725642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1a0ea-c938-4f33-a437-76168a42a29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:29.744786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-aca1a0ea-c938-4f33-a437-76168a42a29f None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:29.745998 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-aca1a0ea-c938-4f33-a437-76168a42a29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:29.753910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1231/2424] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:29.757907 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828ae61-cd4b-4b1d-929d-4fe79c7a7df2 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:29.767991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828ae61-cd4b-4b1d-929d-4fe79c7a7df2 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:29.772716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:29.773371 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:29.773587 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:29.773878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:29.774920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:46:29.774920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:29.776974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:29.788717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1194/2425] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:46:29.792987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1a0ea-c938-4f33-a437-76168a42a29f None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:29.802305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1a0ea-c938-4f33-a437-76168a42a29f None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:29.807038 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75329d0a-454a-41a2-8d2b-e52d60590451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:29.807627 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329d0a-454a-41a2-8d2b-e52d60590451 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:29.807931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329d0a-454a-41a2-8d2b-e52d60590451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:29.808184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329d0a-454a-41a2-8d2b-e52d60590451 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:29.808846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75329d0a-454a-41a2-8d2b-e52d60590451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:29.810819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75329d0a-454a-41a2-8d2b-e52d60590451 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:29.888957 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1232/2426] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:46:29 2022] POST /identity/v3/projects => generated 399 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:46:29.892949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:29.903198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c4dd0-7808-4956-8526-6df498b9cb03 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:29.921878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:29.959772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b30d0f-daa5-4ff0-aaf1-34d189d36ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:29.960372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b30d0f-daa5-4ff0-aaf1-34d189d36ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:29.960654 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b30d0f-daa5-4ff0-aaf1-34d189d36ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:29.960925 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b30d0f-daa5-4ff0-aaf1-34d189d36ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:29.961553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b30d0f-daa5-4ff0-aaf1-34d189d36ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:29.963523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b30d0f-daa5-4ff0-aaf1-34d189d36ca2 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:30.031102 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1195/2427] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:46:29 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:46:30.035258 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75329d0a-454a-41a2-8d2b-e52d60590451 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:30.045731 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75329d0a-454a-41a2-8d2b-e52d60590451 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:30.050335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fda21-abf5-4e24-9ce5-0e85123d122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:30.050928 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fda21-abf5-4e24-9ce5-0e85123d122c admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:30.051210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fda21-abf5-4e24-9ce5-0e85123d122c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:30.051513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fda21-abf5-4e24-9ce5-0e85123d122c admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:30.052151 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fda21-abf5-4e24-9ce5-0e85123d122c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:30.053737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fda21-abf5-4e24-9ce5-0e85123d122c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:30.064161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1233/2428] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:46:30.074216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b30d0f-daa5-4ff0-aaf1-34d189d36ca2 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:30.084632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b30d0f-daa5-4ff0-aaf1-34d189d36ca2 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:30.089350 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fe72b-bee9-4ba6-b9d1-67c620dacc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:30.090027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fe72b-bee9-4ba6-b9d1-67c620dacc91 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:30.090296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fe72b-bee9-4ba6-b9d1-67c620dacc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:30.090573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fe72b-bee9-4ba6-b9d1-67c620dacc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:30.091232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fe72b-bee9-4ba6-b9d1-67c620dacc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:30.092812 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fe72b-bee9-4ba6-b9d1-67c620dacc91 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:30.103285 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1196/2429] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:46:30.107342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fda21-abf5-4e24-9ce5-0e85123d122c admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:30.117280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fda21-abf5-4e24-9ce5-0e85123d122c admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:30.122436 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:30.123070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:30.123347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:30.123620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] PATH_INFO: `/v3/projects/aed8a3b1d4da46fdb62985522fa6027a/users/1276b56553ca46b8b4ad83ef5b558d66/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:30.132203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:30.138614 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed8a3b1d4da46fdb62985522fa6027a, user_id=1276b56553ca46b8b4ad83ef5b558d66, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:30.145241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:30.196612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1234/2430] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:46:30 2022] PUT /identity/v3/projects/aed8a3b1d4da46fdb62985522fa6027a/users/1276b56553ca46b8b4ad83ef5b558d66/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:30.208441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922117db-e0bb-4bef-85d6-56de95ecc4d4 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:30.208750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922117db-e0bb-4bef-85d6-56de95ecc4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:30.209050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922117db-e0bb-4bef-85d6-56de95ecc4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:30.237438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-922117db-e0bb-4bef-85d6-56de95ecc4d4 None None] MFA Rules not processed for user `1276b56553ca46b8b4ad83ef5b558d66`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:30.238668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-922117db-e0bb-4bef-85d6-56de95ecc4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:30.302855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1197/2431] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:30 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:30.313918 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:30.315645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:30.395748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258af1e-6e5a-493a-8a39-44dcc8e59302 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:30.399431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d2258-2652-4a82-9328-282d02073d27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:30.400664 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d2258-2652-4a82-9328-282d02073d27 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:30.400944 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d2258-2652-4a82-9328-282d02073d27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:30.401213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d2258-2652-4a82-9328-282d02073d27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:30.409644 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d2258-2652-4a82-9328-282d02073d27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:30.411128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d2258-2652-4a82-9328-282d02073d27 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:30.420282 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1235/2432] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:38.222394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922117db-e0bb-4bef-85d6-56de95ecc4d4 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:38.284317 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922117db-e0bb-4bef-85d6-56de95ecc4d4 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:38.288816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:38.289439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:38.289720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:38.290032 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] PATH_INFO: `/v3/users/1276b56553ca46b8b4ad83ef5b558d66` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:38.291152 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1276b56553ca46b8b4ad83ef5b558d66)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:38.293094 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:38.450119 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:38.462716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1276b56553ca46b8b4ad83ef5b558d66'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:38.475817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1276b56553ca46b8b4ad83ef5b558d66'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:38.545151 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1276b56553ca46b8b4ad83ef5b558d66'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:38.545787 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1276b56553ca46b8b4ad83ef5b558d66'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:38.552965 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1276b56553ca46b8b4ad83ef5b558d66'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:38.558921 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1198/2433] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:46:38 2022] DELETE /identity/v3/users/1276b56553ca46b8b4ad83ef5b558d66 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:38.563431 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599f7e0-32e5-475a-a83c-0c188c71d586 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:38.563970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599f7e0-32e5-475a-a83c-0c188c71d586 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:38.564082 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599f7e0-32e5-475a-a83c-0c188c71d586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:38.583605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7599f7e0-32e5-475a-a83c-0c188c71d586 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:38.584812 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7599f7e0-32e5-475a-a83c-0c188c71d586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:38.644286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1236/2434] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:38.655090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:38.656690 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:38.713655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d7403-6e15-4029-a90c-5fe1f75db0ae admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:38.717299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e9e03-dad8-4356-bb35-bd3a970ca65d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:38.717896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e9e03-dad8-4356-bb35-bd3a970ca65d service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:38.718201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e9e03-dad8-4356-bb35-bd3a970ca65d service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:38.718467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e9e03-dad8-4356-bb35-bd3a970ca65d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:38.727287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e9e03-dad8-4356-bb35-bd3a970ca65d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:38.728728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e9e03-dad8-4356-bb35-bd3a970ca65d service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:38.737919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1199/2435] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:39.130547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599f7e0-32e5-475a-a83c-0c188c71d586 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:39.132121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7599f7e0-32e5-475a-a83c-0c188c71d586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:39.145400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599f7e0-32e5-475a-a83c-0c188c71d586 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:39.150075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:39.150707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:39.150984 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:39.151266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] PATH_INFO: `/v3/projects/aed8a3b1d4da46fdb62985522fa6027a` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:39.152382 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed8a3b1d4da46fdb62985522fa6027a)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:39.155056 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:39.266340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed8a3b1d4da46fdb62985522fa6027a', 'request_id': 'req-6602de19-bd26-4a66-8093-699af3a83470'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:39.271782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed8a3b1d4da46fdb62985522fa6027a', 'request_id': 'req-6602de19-bd26-4a66-8093-699af3a83470'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:39.278127 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed8a3b1d4da46fdb62985522fa6027a', 'request_id': 'req-6602de19-bd26-4a66-8093-699af3a83470'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:39.281214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:39.367251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] The token cache is being invalidate because project aed8a3b1d4da46fdb62985522fa6027a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:46:39.367535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:39.368148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:46:39.369716 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1237/2436] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:46:39 2022] DELETE /identity/v3/projects/aed8a3b1d4da46fdb62985522fa6027a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:46:48.755959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=43,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:49.414346 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:52.506301 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb4a19-3a65-4e5b-a7b6-f922c43fd474 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:52.506793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb4a19-3a65-4e5b-a7b6-f922c43fd474 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:52.506901 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb4a19-3a65-4e5b-a7b6-f922c43fd474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:52.525383 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-22fb4a19-3a65-4e5b-a7b6-f922c43fd474 None None] MFA Rules not processed for user `205b19ccb80a45fd8ed36abd2c4c32c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:52.526600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22fb4a19-3a65-4e5b-a7b6-f922c43fd474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:52.579855 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1200/2437] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:46:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:47:02.573249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=25 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:04.160703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:04.162136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:04.217915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602de19-bd26-4a66-8093-699af3a83470 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:04.221472 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:04.222092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:04.222378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:04.222651 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:04.224330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:04.310423 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:04.311848 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:04.332501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1238/2438] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:05.508584 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb4a19-3a65-4e5b-a7b6-f922c43fd474 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:05.510006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-22fb4a19-3a65-4e5b-a7b6-f922c43fd474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:05.522180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb4a19-3a65-4e5b-a7b6-f922c43fd474 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:05.525630 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:05.526251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:05.526529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:05.526796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:05.528421 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:05.541725 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:05.543196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:05.553765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1201/2439] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:14.329297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=66 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:15.548932 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=6 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:51.111383 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:51.112777 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:51.168782 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261be5c3-7350-4355-b248-5f14c54f03b8 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:51.172541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426dfd7-e328-4049-b084-89624d136ad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:51.173132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426dfd7-e328-4049-b084-89624d136ad3 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:51.173413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426dfd7-e328-4049-b084-89624d136ad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:51.173687 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426dfd7-e328-4049-b084-89624d136ad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:51.175677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d426dfd7-e328-4049-b084-89624d136ad3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:51.187004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426dfd7-e328-4049-b084-89624d136ad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:51.188414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426dfd7-e328-4049-b084-89624d136ad3 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:51.197293 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1239/2440] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:52.253431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:52.256122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:52.270885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89127682-16b5-46a3-978f-b229d27d6c41 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:52.276621 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:52.277353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:52.277506 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:52.277760 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:52.279514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:52.292132 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:52.293661 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:52.306014 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1202/2441] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:59.166398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfadaf-e99c-4750-98ee-a0b098b0a3ba None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:59.166884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfadaf-e99c-4750-98ee-a0b098b0a3ba None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:59.166995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfadaf-e99c-4750-98ee-a0b098b0a3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:59.187703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-c3cfadaf-e99c-4750-98ee-a0b098b0a3ba None None] MFA Rules not processed for user `f84e6adb1a0e4d709b2962c9793346dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:59.195199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-c3cfadaf-e99c-4750-98ee-a0b098b0a3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:59.242137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1240/2442] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:48:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:48:59.928998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:59.930491 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:00.002332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f87e6-4032-467d-bcfe-411d1a129914 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:00.008279 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:00.008885 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:00.009167 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:00.009441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:00.011159 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:00.022287 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:00.023791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:00.033564 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1203/2443] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:00.633879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b5604-1367-47e4-9b90-f8f6c6e820bb None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:00.634492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b5604-1367-47e4-9b90-f8f6c6e820bb None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:00.634492 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b5604-1367-47e4-9b90-f8f6c6e820bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:00.654048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7d4b5604-1367-47e4-9b90-f8f6c6e820bb None None] MFA Rules not processed for user `205b19ccb80a45fd8ed36abd2c4c32c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:00.655253 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7d4b5604-1367-47e4-9b90-f8f6c6e820bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:00.662991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1241/2444] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:49:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:03.312851 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:03.322372 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3041d-ce2d-4c59-8d65-136081fd8a98 service placement] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:03.325729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:03.326356 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:03.326632 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:03.326897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:03.328791 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:03.339144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:03.340593 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:03.349552 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1204/2445] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:49:05.723396 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b5604-1367-47e4-9b90-f8f6c6e820bb None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:05.726181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7d4b5604-1367-47e4-9b90-f8f6c6e820bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:05.744199 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b5604-1367-47e4-9b90-f8f6c6e820bb None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:05.752081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:05.752702 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:05.752991 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:05.753266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:05.755400 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:05.769752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:05.771921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:05.783166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1242/2446] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:08.826835 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:08.840770 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d605fea-ec78-4c21-b81c-a0ab6c47ac68 service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:08.846333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0a7230c6da74e6484db3f260b4c58df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:08.846922 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.847201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.847484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.849739 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:08.862106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:08.863706 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:08.874234 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1205/2447] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:15.778559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=60 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:18.869711 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=50 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:26.444506 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:26.445987 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:26.513443 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e65b99-c0e8-4132-bcad-8fc86347e3e9 service nova] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:26.518667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:26.519303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:26.519611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:26.519890 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] PATH_INFO: `/v3/users/f84e6adb1a0e4d709b2962c9793346dc` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:26.521015 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84e6adb1a0e4d709b2962c9793346dc)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:26.523327 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:26.735340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84e6adb1a0e4d709b2962c9793346dc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:26.747347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84e6adb1a0e4d709b2962c9793346dc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:26.748032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84e6adb1a0e4d709b2962c9793346dc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:26.826626 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84e6adb1a0e4d709b2962c9793346dc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:26.834203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bd54ceed-9f56-4ddf-be84-d833835a07e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84e6adb1a0e4d709b2962c9793346dc'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:26.840230 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1243/2448] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:26 2022] DELETE /identity/v3/users/f84e6adb1a0e4d709b2962c9793346dc => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:27.083296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.094694 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccec22-cdb6-4f2f-ae0f-5c03cd8fca86 service nova] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.158082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.158717 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.158998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.159273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] PATH_INFO: `/v3/projects/3ebab900aef740b68af799c4695e6e5f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.160388 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ebab900aef740b68af799c4695e6e5f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.162315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.264104 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ebab900aef740b68af799c4695e6e5f', 'request_id': 'req-577f7730-212e-4523-af98-ba627aee1dc4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.270229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ebab900aef740b68af799c4695e6e5f', 'request_id': 'req-577f7730-212e-4523-af98-ba627aee1dc4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.276429 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ebab900aef740b68af799c4695e6e5f', 'request_id': 'req-577f7730-212e-4523-af98-ba627aee1dc4'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.343227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] The token cache is being invalidate because project 3ebab900aef740b68af799c4695e6e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:27.343456 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.344077 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-577f7730-212e-4523-af98-ba627aee1dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.345675 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1206/2449] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:27 2022] DELETE /identity/v3/projects/3ebab900aef740b68af799c4695e6e5f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:27.350769 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefab8d-7ccd-4b92-a0de-adf31b0f2e1e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.351086 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefab8d-7ccd-4b92-a0de-adf31b0f2e1e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.351357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefab8d-7ccd-4b92-a0de-adf31b0f2e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.372117 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-7aefab8d-7ccd-4b92-a0de-adf31b0f2e1e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:27.376729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7aefab8d-7ccd-4b92-a0de-adf31b0f2e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:27.431783 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1244/2450] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:27.439679 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6089cb-0303-4d23-abfd-5615f3bff70a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.439964 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6089cb-0303-4d23-abfd-5615f3bff70a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.440239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6089cb-0303-4d23-abfd-5615f3bff70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.459620 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fe6089cb-0303-4d23-abfd-5615f3bff70a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:27.460806 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fe6089cb-0303-4d23-abfd-5615f3bff70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:27.469049 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1207/2451] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:27.473166 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefab8d-7ccd-4b92-a0de-adf31b0f2e1e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.482845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefab8d-7ccd-4b92-a0de-adf31b0f2e1e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.488045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.488639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.488921 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.489195 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.489837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:50:27.490181 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.491774 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.503573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1245/2452] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:50:27.508216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6089cb-0303-4d23-abfd-5615f3bff70a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.517767 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6089cb-0303-4d23-abfd-5615f3bff70a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.522349 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e5e4f-c542-4255-8a02-30150d6d179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.522937 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e5e4f-c542-4255-8a02-30150d6d179e admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.523212 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e5e4f-c542-4255-8a02-30150d6d179e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.523481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e5e4f-c542-4255-8a02-30150d6d179e admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.524108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e5e4f-c542-4255-8a02-30150d6d179e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.525995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e5e4f-c542-4255-8a02-30150d6d179e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.607498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1208/2453] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:50:27 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:27.611649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.621535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783200fe-7a20-409e-8915-a20d1e8f8e27 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.625189 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:27.679132 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa13ca-9adf-4b1a-89b2-25635f319922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.679738 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa13ca-9adf-4b1a-89b2-25635f319922 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.680014 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa13ca-9adf-4b1a-89b2-25635f319922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.680289 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa13ca-9adf-4b1a-89b2-25635f319922 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.680916 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa13ca-9adf-4b1a-89b2-25635f319922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.682876 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa13ca-9adf-4b1a-89b2-25635f319922 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.794963 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1246/2454] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:50:27 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:27.798920 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e5e4f-c542-4255-8a02-30150d6d179e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.809257 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e5e4f-c542-4255-8a02-30150d6d179e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.814117 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469542-dbe9-439f-b1ca-20bb584b0147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.816493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469542-dbe9-439f-b1ca-20bb584b0147 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.816493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469542-dbe9-439f-b1ca-20bb584b0147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.816493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469542-dbe9-439f-b1ca-20bb584b0147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.816493 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20469542-dbe9-439f-b1ca-20bb584b0147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.817433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20469542-dbe9-439f-b1ca-20bb584b0147 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.827893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1209/2455] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:27.831797 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa13ca-9adf-4b1a-89b2-25635f319922 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.841766 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa13ca-9adf-4b1a-89b2-25635f319922 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.847891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f60b8-9ef5-45a4-be28-e66dc48622fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.848483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f60b8-9ef5-45a4-be28-e66dc48622fe admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.848765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f60b8-9ef5-45a4-be28-e66dc48622fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.849040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f60b8-9ef5-45a4-be28-e66dc48622fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.849678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f60b8-9ef5-45a4-be28-e66dc48622fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.851306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f60b8-9ef5-45a4-be28-e66dc48622fe admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.861192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1247/2456] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:27.865121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469542-dbe9-439f-b1ca-20bb584b0147 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.874980 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469542-dbe9-439f-b1ca-20bb584b0147 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.879553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb9e75-d0e5-4062-9f1a-7e315f75b20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.880185 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9e75-d0e5-4062-9f1a-7e315f75b20e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.880466 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9e75-d0e5-4062-9f1a-7e315f75b20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.880738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9e75-d0e5-4062-9f1a-7e315f75b20e admin admin] PATH_INFO: `/v3/projects/7155e69c1619451eb8aff2996974e347/users/43d772c8ee8e462da100d7bc02b27b40/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.895904 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb9e75-d0e5-4062-9f1a-7e315f75b20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7155e69c1619451eb8aff2996974e347, user_id=43d772c8ee8e462da100d7bc02b27b40, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.898586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb9e75-d0e5-4062-9f1a-7e315f75b20e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.960511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1210/2457] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:50:27 2022] PUT /identity/v3/projects/7155e69c1619451eb8aff2996974e347/users/43d772c8ee8e462da100d7bc02b27b40/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:27.970747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999ff6-9560-478b-9605-242735918a31 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.971038 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999ff6-9560-478b-9605-242735918a31 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.971309 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94999ff6-9560-478b-9605-242735918a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.998457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-94999ff6-9560-478b-9605-242735918a31 None None] MFA Rules not processed for user `43d772c8ee8e462da100d7bc02b27b40`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:28.000141 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-94999ff6-9560-478b-9605-242735918a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:28.062437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1248/2458] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:28.066905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb9e75-d0e5-4062-9f1a-7e315f75b20e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:28.087223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:28.129098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb9e75-d0e5-4062-9f1a-7e315f75b20e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:28.134794 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117e504-045e-4dde-90de-f6b040082ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:28.135997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117e504-045e-4dde-90de-f6b040082ece admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:28.135997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117e504-045e-4dde-90de-f6b040082ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:28.135997 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117e504-045e-4dde-90de-f6b040082ece admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:28.136585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117e504-045e-4dde-90de-f6b040082ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:28.138515 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117e504-045e-4dde-90de-f6b040082ece admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:28.247366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1211/2459] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:50:28 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:50:28.251925 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999ff6-9560-478b-9605-242735918a31 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:28.262985 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94999ff6-9560-478b-9605-242735918a31 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:28.319592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc11a9c-6565-4652-bad9-52912bfcd105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:28.320193 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc11a9c-6565-4652-bad9-52912bfcd105 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:28.320474 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc11a9c-6565-4652-bad9-52912bfcd105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:28.320754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc11a9c-6565-4652-bad9-52912bfcd105 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:28.321377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc11a9c-6565-4652-bad9-52912bfcd105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:28.323338 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc11a9c-6565-4652-bad9-52912bfcd105 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:28.417609 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1249/2460] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:50:28 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:28.421651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117e504-045e-4dde-90de-f6b040082ece admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:28.432165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117e504-045e-4dde-90de-f6b040082ece admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:28.436808 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8db727-4b9c-40d0-86f4-d418145b6b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:28.437404 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8db727-4b9c-40d0-86f4-d418145b6b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:28.437687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8db727-4b9c-40d0-86f4-d418145b6b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:28.437985 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8db727-4b9c-40d0-86f4-d418145b6b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:28.438629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8db727-4b9c-40d0-86f4-d418145b6b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:28.440260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8db727-4b9c-40d0-86f4-d418145b6b17 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:28.450631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1212/2461] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:28.455205 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc11a9c-6565-4652-bad9-52912bfcd105 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:28.461525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:28.465375 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc11a9c-6565-4652-bad9-52912bfcd105 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:28.470691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80553b99-44ef-4ddc-b0ef-90b38c0cf57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:28.471333 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80553b99-44ef-4ddc-b0ef-90b38c0cf57f admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:28.472239 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80553b99-44ef-4ddc-b0ef-90b38c0cf57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:28.472521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80553b99-44ef-4ddc-b0ef-90b38c0cf57f admin admin] PATH_INFO: `/v3/projects/c75b255b46304df5b83380a3ac351935/users/3978750947084d008da6fc4a9f82b9f7/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:28.487576 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80553b99-44ef-4ddc-b0ef-90b38c0cf57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75b255b46304df5b83380a3ac351935, user_id=3978750947084d008da6fc4a9f82b9f7, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:28.490245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80553b99-44ef-4ddc-b0ef-90b38c0cf57f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:28.578215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1250/2462] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:50:28 2022] PUT /identity/v3/projects/c75b255b46304df5b83380a3ac351935/users/3978750947084d008da6fc4a9f82b9f7/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:28.582224 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8db727-4b9c-40d0-86f4-d418145b6b17 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:28.592290 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8db727-4b9c-40d0-86f4-d418145b6b17 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:28.648445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629499a5-4666-4942-9771-8ab93ebf6e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:28.649035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629499a5-4666-4942-9771-8ab93ebf6e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:28.649332 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629499a5-4666-4942-9771-8ab93ebf6e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:28.649609 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629499a5-4666-4942-9771-8ab93ebf6e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:28.650271 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629499a5-4666-4942-9771-8ab93ebf6e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:28.651876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629499a5-4666-4942-9771-8ab93ebf6e88 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:28.661820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1213/2463] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:28.666050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80553b99-44ef-4ddc-b0ef-90b38c0cf57f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:28.676471 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80553b99-44ef-4ddc-b0ef-90b38c0cf57f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:28.681081 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:28.681722 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:28.682030 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:28.682304 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] PATH_INFO: `/v3/projects/c75b255b46304df5b83380a3ac351935/users/3978750947084d008da6fc4a9f82b9f7/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:28.684196 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75b255b46304df5b83380a3ac351935, user_id=3978750947084d008da6fc4a9f82b9f7, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:28.687439 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:28.697557 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1251/2464] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:50:28 2022] PUT /identity/v3/projects/c75b255b46304df5b83380a3ac351935/users/3978750947084d008da6fc4a9f82b9f7/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:28.713523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9fd44-f85a-4459-b152-e3019a9d1b30 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:28.713810 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9fd44-f85a-4459-b152-e3019a9d1b30 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:28.714117 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9fd44-f85a-4459-b152-e3019a9d1b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:28.743716 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-2de9fd44-f85a-4459-b152-e3019a9d1b30 None None] MFA Rules not processed for user `3978750947084d008da6fc4a9f82b9f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:28.744909 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-2de9fd44-f85a-4459-b152-e3019a9d1b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:28.811554 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1214/2465] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:28.822229 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:28.823969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:28.881368 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db57131-e762-44f8-a57b-f203944d80e9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:28.885207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd155b-4add-4a4c-8a3e-5eee9ecb0ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:28.885792 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd155b-4add-4a4c-8a3e-5eee9ecb0ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:28.886109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd155b-4add-4a4c-8a3e-5eee9ecb0ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:28.886377 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd155b-4add-4a4c-8a3e-5eee9ecb0ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:28.895706 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd155b-4add-4a4c-8a3e-5eee9ecb0ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:28.897109 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd155b-4add-4a4c-8a3e-5eee9ecb0ad8 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:28.906449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1252/2466] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:29.009249 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9fd44-f85a-4459-b152-e3019a9d1b30 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:29.019400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9fd44-f85a-4459-b152-e3019a9d1b30 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:29.022928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69359dca-1978-421d-b156-75e435f9377d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:29.023527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359dca-1978-421d-b156-75e435f9377d service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:29.023802 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359dca-1978-421d-b156-75e435f9377d service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:29.024070 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359dca-1978-421d-b156-75e435f9377d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:29.033198 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:29.081708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69359dca-1978-421d-b156-75e435f9377d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:29.083274 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69359dca-1978-421d-b156-75e435f9377d service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:29.092703 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1215/2467] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:32.109178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd155b-4add-4a4c-8a3e-5eee9ecb0ad8 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:32.171280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd155b-4add-4a4c-8a3e-5eee9ecb0ad8 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:32.176246 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:32.176861 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:32.177139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:32.177408 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] PATH_INFO: `/v3/users/43d772c8ee8e462da100d7bc02b27b40` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:32.178528 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d772c8ee8e462da100d7bc02b27b40)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:32.180445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:32.380955 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d772c8ee8e462da100d7bc02b27b40'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:32.393078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d772c8ee8e462da100d7bc02b27b40'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:32.393734 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d772c8ee8e462da100d7bc02b27b40'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:32.454945 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d772c8ee8e462da100d7bc02b27b40'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:32.456167 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:32.462409 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d772c8ee8e462da100d7bc02b27b40'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:32.468251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1253/2468] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:32 2022] DELETE /identity/v3/users/43d772c8ee8e462da100d7bc02b27b40 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:32.473529 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a2d28-219c-4b0b-8ad4-1bcfc7d4d3fc None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:32.474037 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a2d28-219c-4b0b-8ad4-1bcfc7d4d3fc None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:32.474174 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a2d28-219c-4b0b-8ad4-1bcfc7d4d3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:32.493729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-fe6a2d28-219c-4b0b-8ad4-1bcfc7d4d3fc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:32.494966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fe6a2d28-219c-4b0b-8ad4-1bcfc7d4d3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:32.554121 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1216/2469] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:32.563694 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:32.565103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:32.622036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d443e06-1453-4526-83a4-0bba82c1ac85 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:32.625684 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf7e6-7404-4892-82b0-91deb9db99b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:32.626312 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf7e6-7404-4892-82b0-91deb9db99b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:32.630060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf7e6-7404-4892-82b0-91deb9db99b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:32.630060 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf7e6-7404-4892-82b0-91deb9db99b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:32.636490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cf7e6-7404-4892-82b0-91deb9db99b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:32.637836 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cf7e6-7404-4892-82b0-91deb9db99b2 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:32.647192 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1254/2470] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:33.074088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a2d28-219c-4b0b-8ad4-1bcfc7d4d3fc None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:33.075516 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-fe6a2d28-219c-4b0b-8ad4-1bcfc7d4d3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:33.088308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a2d28-219c-4b0b-8ad4-1bcfc7d4d3fc None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:33.092786 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:33.093417 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:33.093695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:33.093998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] PATH_INFO: `/v3/projects/7155e69c1619451eb8aff2996974e347` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:33.095683 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7155e69c1619451eb8aff2996974e347)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:33.097598 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:33.191771 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7155e69c1619451eb8aff2996974e347', 'request_id': 'req-a92be550-fcfc-45c7-906a-afc02b911950'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.197155 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7155e69c1619451eb8aff2996974e347', 'request_id': 'req-a92be550-fcfc-45c7-906a-afc02b911950'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.203338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7155e69c1619451eb8aff2996974e347', 'request_id': 'req-a92be550-fcfc-45c7-906a-afc02b911950'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.293887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] The token cache is being invalidate because project 7155e69c1619451eb8aff2996974e347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:33.294202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.294817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.296292 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1217/2471] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:33 2022] DELETE /identity/v3/projects/7155e69c1619451eb8aff2996974e347 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:33.301072 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf7e6-7404-4892-82b0-91deb9db99b2 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:33.310932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf7e6-7404-4892-82b0-91deb9db99b2 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:33.367310 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:33.367932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:33.368212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:33.368512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] PATH_INFO: `/v3/users/3978750947084d008da6fc4a9f82b9f7` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:33.369621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3978750947084d008da6fc4a9f82b9f7)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:33.371585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:33.514554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3978750947084d008da6fc4a9f82b9f7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.527298 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3978750947084d008da6fc4a9f82b9f7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.528009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3978750947084d008da6fc4a9f82b9f7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.690511 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3978750947084d008da6fc4a9f82b9f7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.698459 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-976bf3ec-9a1c-4919-b53e-13c0169f03f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3978750947084d008da6fc4a9f82b9f7'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:33.704350 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1255/2472] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:33 2022] DELETE /identity/v3/users/3978750947084d008da6fc4a9f82b9f7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:34.108289 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:34.109708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:34.122949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92be550-fcfc-45c7-906a-afc02b911950 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:34.157184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:34.179952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:34.180569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.180848 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.181116 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] PATH_INFO: `/v3/projects/c75b255b46304df5b83380a3ac351935` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.182239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75b255b46304df5b83380a3ac351935)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:34.184122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:34.291828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75b255b46304df5b83380a3ac351935', 'request_id': 'req-5eddb040-5c00-4e3b-b67a-f02ef19d8222'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:34.297437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75b255b46304df5b83380a3ac351935', 'request_id': 'req-5eddb040-5c00-4e3b-b67a-f02ef19d8222'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:34.304875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75b255b46304df5b83380a3ac351935', 'request_id': 'req-5eddb040-5c00-4e3b-b67a-f02ef19d8222'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:34.383284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] The token cache is being invalidate because project c75b255b46304df5b83380a3ac351935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:34.383494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:34.384142 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-5eddb040-5c00-4e3b-b67a-f02ef19d8222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:34.385669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1218/2473] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:34 2022] DELETE /identity/v3/projects/c75b255b46304df5b83380a3ac351935 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:34.390910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60341251-5c60-449e-95d9-6852f11e94ef None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.391428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60341251-5c60-449e-95d9-6852f11e94ef None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.392041 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60341251-5c60-449e-95d9-6852f11e94ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.412068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-60341251-5c60-449e-95d9-6852f11e94ef None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:34.413268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-60341251-5c60-449e-95d9-6852f11e94ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:34.418543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:34.471567 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1256/2474] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:34.479096 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13959a4-776b-43ef-bf4a-0aa456e8d0af None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.479397 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13959a4-776b-43ef-bf4a-0aa456e8d0af None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.479678 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13959a4-776b-43ef-bf4a-0aa456e8d0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.499738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-d13959a4-776b-43ef-bf4a-0aa456e8d0af None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:34.501499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-d13959a4-776b-43ef-bf4a-0aa456e8d0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:34.509697 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1219/2475] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:34.513979 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60341251-5c60-449e-95d9-6852f11e94ef None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:34.523437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60341251-5c60-449e-95d9-6852f11e94ef None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:34.528790 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:34.529381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.529662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.529962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.530593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:50:34.530896 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:34.532466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:34.544118 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1257/2476] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:50:34.548720 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13959a4-776b-43ef-bf4a-0aa456e8d0af None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:34.557823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13959a4-776b-43ef-bf4a-0aa456e8d0af None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:34.562492 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8ec0-3fab-47fd-b505-368fdb309177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:34.563082 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8ec0-3fab-47fd-b505-368fdb309177 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.563363 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8ec0-3fab-47fd-b505-368fdb309177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.564229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8ec0-3fab-47fd-b505-368fdb309177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.564868 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e8ec0-3fab-47fd-b505-368fdb309177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:34.566770 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e8ec0-3fab-47fd-b505-368fdb309177 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:34.647766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1220/2477] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:50:34 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:34.651578 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:34.662336 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96eb52-d4c3-492c-aede-dac6a688961b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:34.717378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6368ac1-f2dc-47b0-88c3-035b24f8e0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:34.717995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6368ac1-f2dc-47b0-88c3-035b24f8e0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.718276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6368ac1-f2dc-47b0-88c3-035b24f8e0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.718546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6368ac1-f2dc-47b0-88c3-035b24f8e0c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.719157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6368ac1-f2dc-47b0-88c3-035b24f8e0c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:34.721572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6368ac1-f2dc-47b0-88c3-035b24f8e0c9 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:34.829635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1258/2478] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:50:34 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:34.830976 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8ec0-3fab-47fd-b505-368fdb309177 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:34.841284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8ec0-3fab-47fd-b505-368fdb309177 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:34.846487 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf79b-5516-4584-9d15-0039b1af6ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:34.847084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf79b-5516-4584-9d15-0039b1af6ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.847367 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf79b-5516-4584-9d15-0039b1af6ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.847639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf79b-5516-4584-9d15-0039b1af6ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.848341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf79b-5516-4584-9d15-0039b1af6ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:34.849919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf79b-5516-4584-9d15-0039b1af6ee8 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:34.859635 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1221/2479] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:34.864413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6368ac1-f2dc-47b0-88c3-035b24f8e0c9 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:34.874407 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6368ac1-f2dc-47b0-88c3-035b24f8e0c9 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:34.878946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47dba-939b-41b6-8a53-e68185365991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:34.879551 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47dba-939b-41b6-8a53-e68185365991 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.879827 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47dba-939b-41b6-8a53-e68185365991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.880097 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47dba-939b-41b6-8a53-e68185365991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.880725 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47dba-939b-41b6-8a53-e68185365991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:34.882593 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47dba-939b-41b6-8a53-e68185365991 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:34.892911 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1259/2480] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:34.896785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf79b-5516-4584-9d15-0039b1af6ee8 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:34.906601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf79b-5516-4584-9d15-0039b1af6ee8 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:34.911604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ad1f-64c1-43a1-82fc-dc15df94a7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:34.912233 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ad1f-64c1-43a1-82fc-dc15df94a7ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:34.912511 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ad1f-64c1-43a1-82fc-dc15df94a7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:34.912783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ad1f-64c1-43a1-82fc-dc15df94a7ea admin admin] PATH_INFO: `/v3/projects/c7b0150bbbd64f9aa29fa4265cb40daa/users/0d576a603ea14f7795f7ba01975f004d/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:34.927586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094ad1f-64c1-43a1-82fc-dc15df94a7ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b0150bbbd64f9aa29fa4265cb40daa, user_id=0d576a603ea14f7795f7ba01975f004d, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:34.930240 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094ad1f-64c1-43a1-82fc-dc15df94a7ea admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:35.018651 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1222/2481] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:50:34 2022] PUT /identity/v3/projects/c7b0150bbbd64f9aa29fa4265cb40daa/users/0d576a603ea14f7795f7ba01975f004d/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:35.029048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde03501-bb9f-436c-b6fb-2021185aeb6e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.029329 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde03501-bb9f-436c-b6fb-2021185aeb6e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.029603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde03501-bb9f-436c-b6fb-2021185aeb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.057388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-cde03501-bb9f-436c-b6fb-2021185aeb6e None None] MFA Rules not processed for user `0d576a603ea14f7795f7ba01975f004d`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:35.058667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-cde03501-bb9f-436c-b6fb-2021185aeb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:35.123168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1260/2482] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:35.127673 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ad1f-64c1-43a1-82fc-dc15df94a7ea admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:35.188315 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ad1f-64c1-43a1-82fc-dc15df94a7ea admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:35.192816 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6531e9c-cdbe-4549-a003-eb181e5fca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:35.193411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531e9c-cdbe-4549-a003-eb181e5fca05 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.193693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531e9c-cdbe-4549-a003-eb181e5fca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.193991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6531e9c-cdbe-4549-a003-eb181e5fca05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.194624 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6531e9c-cdbe-4549-a003-eb181e5fca05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:35.196512 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6531e9c-cdbe-4549-a003-eb181e5fca05 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:35.201668 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:35.328666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1223/2483] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:50:35 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:35.333263 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde03501-bb9f-436c-b6fb-2021185aeb6e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:35.344090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde03501-bb9f-436c-b6fb-2021185aeb6e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:35.347547 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:35.399655 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83bea4-c052-4b21-ad32-97f774140f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:35.400299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83bea4-c052-4b21-ad32-97f774140f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.400523 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83bea4-c052-4b21-ad32-97f774140f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.400794 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83bea4-c052-4b21-ad32-97f774140f06 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.401425 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83bea4-c052-4b21-ad32-97f774140f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:35.403380 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83bea4-c052-4b21-ad32-97f774140f06 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:35.492721 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1261/2484] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:50:35 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:35.496605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6531e9c-cdbe-4549-a003-eb181e5fca05 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:35.507692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6531e9c-cdbe-4549-a003-eb181e5fca05 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:35.512305 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b1b14-6d11-44e1-9106-94e8696b32cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:35.512893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b1b14-6d11-44e1-9106-94e8696b32cb admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.513166 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b1b14-6d11-44e1-9106-94e8696b32cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.513434 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b1b14-6d11-44e1-9106-94e8696b32cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.514106 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b1b14-6d11-44e1-9106-94e8696b32cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:35.515680 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b1b14-6d11-44e1-9106-94e8696b32cb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:35.525970 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1224/2485] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:35.529569 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83bea4-c052-4b21-ad32-97f774140f06 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:35.539483 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83bea4-c052-4b21-ad32-97f774140f06 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:35.544070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb40059-293e-49ad-bf5c-4488bc2862f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:35.544707 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb40059-293e-49ad-bf5c-4488bc2862f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.544992 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb40059-293e-49ad-bf5c-4488bc2862f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.545267 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb40059-293e-49ad-bf5c-4488bc2862f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.545977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb40059-293e-49ad-bf5c-4488bc2862f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:35.547572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb40059-293e-49ad-bf5c-4488bc2862f8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:35.557287 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1262/2486] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:35.561169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b1b14-6d11-44e1-9106-94e8696b32cb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:35.571358 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b1b14-6d11-44e1-9106-94e8696b32cb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:35.575859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee1409-2365-4c40-a21b-729f3d4e7a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:35.576485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee1409-2365-4c40-a21b-729f3d4e7a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.576759 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee1409-2365-4c40-a21b-729f3d4e7a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.577027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee1409-2365-4c40-a21b-729f3d4e7a1e admin admin] PATH_INFO: `/v3/projects/e20dff18b86d4f97ad88d8872e1fd04f/users/7de40d6361664a10ab9d1ffd731a99df/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.592090 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee1409-2365-4c40-a21b-729f3d4e7a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20dff18b86d4f97ad88d8872e1fd04f, user_id=7de40d6361664a10ab9d1ffd731a99df, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:35.594743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee1409-2365-4c40-a21b-729f3d4e7a1e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:35.658085 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1225/2487] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:50:35 2022] PUT /identity/v3/projects/e20dff18b86d4f97ad88d8872e1fd04f/users/7de40d6361664a10ab9d1ffd731a99df/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:35.675507 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b086e3-cd29-4bd4-a5b9-368a966012f5 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.675786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b086e3-cd29-4bd4-a5b9-368a966012f5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.676054 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b086e3-cd29-4bd4-a5b9-368a966012f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.705032 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-30b086e3-cd29-4bd4-a5b9-368a966012f5 None None] MFA Rules not processed for user `7de40d6361664a10ab9d1ffd731a99df`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:35.706305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-30b086e3-cd29-4bd4-a5b9-368a966012f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:35.770010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1263/2488] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:35.774403 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee1409-2365-4c40-a21b-729f3d4e7a1e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:35.835612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee1409-2365-4c40-a21b-729f3d4e7a1e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:35.840271 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6ed01-68c0-44a2-b5c9-afe7bb70419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:35.840865 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6ed01-68c0-44a2-b5c9-afe7bb70419b admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.841146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6ed01-68c0-44a2-b5c9-afe7bb70419b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.841419 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6ed01-68c0-44a2-b5c9-afe7bb70419b admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.842088 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6ed01-68c0-44a2-b5c9-afe7bb70419b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:35.843968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6ed01-68c0-44a2-b5c9-afe7bb70419b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:35.919056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1226/2489] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:50:35 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:35.923185 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b086e3-cd29-4bd4-a5b9-368a966012f5 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:35.934603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b086e3-cd29-4bd4-a5b9-368a966012f5 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:35.989271 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0aaa03-af33-4e07-863e-2bc803d93526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:35.989859 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0aaa03-af33-4e07-863e-2bc803d93526 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:35.990161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0aaa03-af33-4e07-863e-2bc803d93526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:35.990430 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0aaa03-af33-4e07-863e-2bc803d93526 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:35.991057 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0aaa03-af33-4e07-863e-2bc803d93526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:35.992952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0aaa03-af33-4e07-863e-2bc803d93526 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:36.095754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1264/2490] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:50:35 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:36.100385 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6ed01-68c0-44a2-b5c9-afe7bb70419b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:36.110793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6ed01-68c0-44a2-b5c9-afe7bb70419b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:36.116024 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ab1e-32a5-4205-8d81-fc1b792965c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:36.116604 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ab1e-32a5-4205-8d81-fc1b792965c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:36.116884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ab1e-32a5-4205-8d81-fc1b792965c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:36.117154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ab1e-32a5-4205-8d81-fc1b792965c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:36.117788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812ab1e-32a5-4205-8d81-fc1b792965c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:36.119398 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812ab1e-32a5-4205-8d81-fc1b792965c0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:36.129805 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1227/2491] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:36.134045 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0aaa03-af33-4e07-863e-2bc803d93526 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:36.140313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:36.144036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0aaa03-af33-4e07-863e-2bc803d93526 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:36.148771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbf37e-693c-487f-9dd6-9a130ab81460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:36.151467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbf37e-693c-487f-9dd6-9a130ab81460 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:36.151467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbf37e-693c-487f-9dd6-9a130ab81460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:36.151467 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbf37e-693c-487f-9dd6-9a130ab81460 admin admin] PATH_INFO: `/v3/projects/491b3d2b22f942b7bfe32be6dafb27b4/users/5ce38f158e2e401dbf60a76ac94da1e3/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:36.164845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbf37e-693c-487f-9dd6-9a130ab81460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=491b3d2b22f942b7bfe32be6dafb27b4, user_id=5ce38f158e2e401dbf60a76ac94da1e3, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:36.167510 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbf37e-693c-487f-9dd6-9a130ab81460 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:36.220703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1265/2492] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:50:36 2022] PUT /identity/v3/projects/491b3d2b22f942b7bfe32be6dafb27b4/users/5ce38f158e2e401dbf60a76ac94da1e3/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:36.225270 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ab1e-32a5-4205-8d81-fc1b792965c0 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:36.235339 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ab1e-32a5-4205-8d81-fc1b792965c0 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:36.292392 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85b826-f2ec-430d-9702-baf24b834801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:36.292998 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85b826-f2ec-430d-9702-baf24b834801 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:36.293280 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85b826-f2ec-430d-9702-baf24b834801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:36.293555 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85b826-f2ec-430d-9702-baf24b834801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:36.294227 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85b826-f2ec-430d-9702-baf24b834801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:36.295825 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85b826-f2ec-430d-9702-baf24b834801 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:36.306324 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1228/2493] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:36.310378 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbf37e-693c-487f-9dd6-9a130ab81460 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:36.320968 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbf37e-693c-487f-9dd6-9a130ab81460 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:36.325877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:36.326621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:36.326910 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:36.327212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] PATH_INFO: `/v3/projects/491b3d2b22f942b7bfe32be6dafb27b4/users/5ce38f158e2e401dbf60a76ac94da1e3/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:36.329122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=491b3d2b22f942b7bfe32be6dafb27b4, user_id=5ce38f158e2e401dbf60a76ac94da1e3, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:36.331793 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:36.342099 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1266/2494] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:50:36 2022] PUT /identity/v3/projects/491b3d2b22f942b7bfe32be6dafb27b4/users/5ce38f158e2e401dbf60a76ac94da1e3/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:36.352769 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13cb34-2519-4bdf-9754-e5b2027d781e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:36.355777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13cb34-2519-4bdf-9754-e5b2027d781e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:36.355777 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13cb34-2519-4bdf-9754-e5b2027d781e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:36.383457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-9b13cb34-2519-4bdf-9754-e5b2027d781e None None] MFA Rules not processed for user `5ce38f158e2e401dbf60a76ac94da1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:36.384669 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9b13cb34-2519-4bdf-9754-e5b2027d781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:36.394165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:36.451941 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1229/2495] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:36.463522 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:36.465221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:36.523071 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7fa97-79e9-4eb5-9f1a-f7ae316f2d0b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:36.526783 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d1461-36ed-48c2-96a7-1e459b8f240c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:36.527388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1461-36ed-48c2-96a7-1e459b8f240c service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:36.527667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1461-36ed-48c2-96a7-1e459b8f240c service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:36.527938 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1461-36ed-48c2-96a7-1e459b8f240c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:36.537607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d1461-36ed-48c2-96a7-1e459b8f240c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:36.539138 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d1461-36ed-48c2-96a7-1e459b8f240c service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:36.548741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1267/2496] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:36.681452 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13cb34-2519-4bdf-9754-e5b2027d781e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:36.682915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9b13cb34-2519-4bdf-9754-e5b2027d781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:36.694646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13cb34-2519-4bdf-9754-e5b2027d781e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:36.698202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c717e-1783-48a3-a577-fc1fa972fd07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:36.698801 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c717e-1783-48a3-a577-fc1fa972fd07 service cinder] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:36.699080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c717e-1783-48a3-a577-fc1fa972fd07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:36.699352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c717e-1783-48a3-a577-fc1fa972fd07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:36.755035 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c717e-1783-48a3-a577-fc1fa972fd07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:36.756460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c717e-1783-48a3-a577-fc1fa972fd07 service cinder] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:36.765539 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1230/2497] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:40.692953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d1461-36ed-48c2-96a7-1e459b8f240c service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:40.755914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d1461-36ed-48c2-96a7-1e459b8f240c service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:40.761085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:40.761711 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:40.762010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:40.762280 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] PATH_INFO: `/v3/users/0d576a603ea14f7795f7ba01975f004d` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:40.763463 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d576a603ea14f7795f7ba01975f004d)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:40.765344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:40.948971 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d576a603ea14f7795f7ba01975f004d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:40.961420 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d576a603ea14f7795f7ba01975f004d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:40.962106 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d576a603ea14f7795f7ba01975f004d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:41.037940 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d576a603ea14f7795f7ba01975f004d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:41.039278 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:41.045685 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d576a603ea14f7795f7ba01975f004d'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:41.051680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1268/2498] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:40 2022] DELETE /identity/v3/users/0d576a603ea14f7795f7ba01975f004d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:50:41.056222 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203717fb-e419-4be0-902b-14ef9bc75032 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:41.056712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203717fb-e419-4be0-902b-14ef9bc75032 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:41.056872 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203717fb-e419-4be0-902b-14ef9bc75032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:41.075775 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-203717fb-e419-4be0-902b-14ef9bc75032 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:41.076968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-203717fb-e419-4be0-902b-14ef9bc75032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:41.135503 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1231/2499] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:41.146136 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:41.147519 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:41.203174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f009be0-5fde-4584-abb9-06e637645a25 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:41.207247 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adff60-8eee-41dc-be02-4eaaf2bc459c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:41.207832 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adff60-8eee-41dc-be02-4eaaf2bc459c service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:41.208137 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adff60-8eee-41dc-be02-4eaaf2bc459c service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:41.208406 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adff60-8eee-41dc-be02-4eaaf2bc459c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:41.216995 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adff60-8eee-41dc-be02-4eaaf2bc459c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:41.218426 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adff60-8eee-41dc-be02-4eaaf2bc459c service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:41.228227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1269/2500] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:41.660133 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203717fb-e419-4be0-902b-14ef9bc75032 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:41.661687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-203717fb-e419-4be0-902b-14ef9bc75032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:41.675072 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203717fb-e419-4be0-902b-14ef9bc75032 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:41.679884 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:41.680437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:41.680718 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:41.680993 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] PATH_INFO: `/v3/projects/c7b0150bbbd64f9aa29fa4265cb40daa` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:41.682148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b0150bbbd64f9aa29fa4265cb40daa)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:41.687519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:41.775394 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:41.781286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b0150bbbd64f9aa29fa4265cb40daa', 'request_id': 'req-b7ba26e9-f559-418c-b9f9-8c4f917c5074'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:41.786719 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b0150bbbd64f9aa29fa4265cb40daa', 'request_id': 'req-b7ba26e9-f559-418c-b9f9-8c4f917c5074'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:41.792968 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b0150bbbd64f9aa29fa4265cb40daa', 'request_id': 'req-b7ba26e9-f559-418c-b9f9-8c4f917c5074'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:41.860415 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] The token cache is being invalidate because project c7b0150bbbd64f9aa29fa4265cb40daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:41.860699 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:41.861333 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:41.863375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1232/2501] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:41 2022] DELETE /identity/v3/projects/c7b0150bbbd64f9aa29fa4265cb40daa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:41.867362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adff60-8eee-41dc-be02-4eaaf2bc459c service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:41.877188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adff60-8eee-41dc-be02-4eaaf2bc459c service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:41.933649 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:41.934225 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:41.934505 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:41.934780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] PATH_INFO: `/v3/users/7de40d6361664a10ab9d1ffd731a99df` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:41.935887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7de40d6361664a10ab9d1ffd731a99df)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:41.937780 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:42.115362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7de40d6361664a10ab9d1ffd731a99df'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.127168 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7de40d6361664a10ab9d1ffd731a99df'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.127822 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7de40d6361664a10ab9d1ffd731a99df'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.198286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7de40d6361664a10ab9d1ffd731a99df'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.205639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7de40d6361664a10ab9d1ffd731a99df'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.211669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1270/2502] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:41 2022] DELETE /identity/v3/users/7de40d6361664a10ab9d1ffd731a99df => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:42.619974 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:42.622859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:42.636899 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba26e9-f559-418c-b9f9-8c4f917c5074 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:42.695531 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:42.696158 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:42.696462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:42.696818 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] PATH_INFO: `/v3/projects/e20dff18b86d4f97ad88d8872e1fd04f` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:42.698025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e20dff18b86d4f97ad88d8872e1fd04f)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:42.699936 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:42.812883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e20dff18b86d4f97ad88d8872e1fd04f', 'request_id': 'req-1a94b1ce-b561-43fd-b859-4d1f5759301e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.819031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e20dff18b86d4f97ad88d8872e1fd04f', 'request_id': 'req-1a94b1ce-b561-43fd-b859-4d1f5759301e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.825346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e20dff18b86d4f97ad88d8872e1fd04f', 'request_id': 'req-1a94b1ce-b561-43fd-b859-4d1f5759301e'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.900649 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] The token cache is being invalidate because project e20dff18b86d4f97ad88d8872e1fd04f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:42.900943 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.901588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:42.903098 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1233/2503] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:42 2022] DELETE /identity/v3/projects/e20dff18b86d4f97ad88d8872e1fd04f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:42.907211 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:42.908673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:42.921284 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b1e3d-deea-4bd6-8efa-803bb24df190 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:42.929110 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:42.978223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:42.978874 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:42.979157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:42.979432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] PATH_INFO: `/v3/users/5ce38f158e2e401dbf60a76ac94da1e3` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:42.980598 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce38f158e2e401dbf60a76ac94da1e3)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:42.983033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:43.180587 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce38f158e2e401dbf60a76ac94da1e3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.193344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce38f158e2e401dbf60a76ac94da1e3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.194040 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce38f158e2e401dbf60a76ac94da1e3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.280817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce38f158e2e401dbf60a76ac94da1e3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.288156 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-e38bf34e-f75b-4219-8c51-bc6d5fd9977b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce38f158e2e401dbf60a76ac94da1e3'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.294128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1271/2504] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:42 2022] DELETE /identity/v3/users/5ce38f158e2e401dbf60a76ac94da1e3 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:43.668817 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:43.670335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:43.683776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94b1ce-b561-43fd-b859-4d1f5759301e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:43.741486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:43.742184 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:43.742460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:43.742729 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] PATH_INFO: `/v3/projects/491b3d2b22f942b7bfe32be6dafb27b4` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:43.743820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=491b3d2b22f942b7bfe32be6dafb27b4)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:43.745698 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:43.747145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:43.867517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '491b3d2b22f942b7bfe32be6dafb27b4', 'request_id': 'req-01df5080-3280-4196-a02a-ada6cdb80db0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.873018 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '491b3d2b22f942b7bfe32be6dafb27b4', 'request_id': 'req-01df5080-3280-4196-a02a-ada6cdb80db0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.879246 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '491b3d2b22f942b7bfe32be6dafb27b4', 'request_id': 'req-01df5080-3280-4196-a02a-ada6cdb80db0'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.954230 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] The token cache is being invalidate because project 491b3d2b22f942b7bfe32be6dafb27b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:43.954448 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.955056 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-01df5080-3280-4196-a02a-ada6cdb80db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:43.957005 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1234/2505] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:43 2022] DELETE /identity/v3/projects/491b3d2b22f942b7bfe32be6dafb27b4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:50:43.962914 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4cee2-922e-4f9a-ac36-cb42d1c5fe8e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:43.963413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4cee2-922e-4f9a-ac36-cb42d1c5fe8e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:43.963512 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4cee2-922e-4f9a-ac36-cb42d1c5fe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:43.984146 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-54f4cee2-922e-4f9a-ac36-cb42d1c5fe8e None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:43.985328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-54f4cee2-922e-4f9a-ac36-cb42d1c5fe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:44.042941 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1272/2506] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:44.104764 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60aeb7-64a0-4281-af78-b2d32055c867 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.105026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60aeb7-64a0-4281-af78-b2d32055c867 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.105294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60aeb7-64a0-4281-af78-b2d32055c867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.124788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bb60aeb7-64a0-4281-af78-b2d32055c867 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:44.126058 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bb60aeb7-64a0-4281-af78-b2d32055c867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:44.134400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1235/2507] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:44.138458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4cee2-922e-4f9a-ac36-cb42d1c5fe8e None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:44.148729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4cee2-922e-4f9a-ac36-cb42d1c5fe8e None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:44.153241 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:44.153831 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.154139 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.154413 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.155070 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:50:44.155360 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:44.156946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:44.168816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1273/2508] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:50:44.172948 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60aeb7-64a0-4281-af78-b2d32055c867 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:44.181987 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60aeb7-64a0-4281-af78-b2d32055c867 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:44.186656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567adaf-1b5d-4d82-a018-b498845cafc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:44.187239 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567adaf-1b5d-4d82-a018-b498845cafc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.187514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567adaf-1b5d-4d82-a018-b498845cafc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.187785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567adaf-1b5d-4d82-a018-b498845cafc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.188411 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567adaf-1b5d-4d82-a018-b498845cafc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:44.190334 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567adaf-1b5d-4d82-a018-b498845cafc5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:44.281670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1236/2509] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:50:44 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:50:44.286437 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:44.296429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c302-e8d5-44c0-a924-2ea9cf05012e admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:44.308752 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:44.352838 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64213f47-32d1-417b-9427-13669ba6ac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:44.353422 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64213f47-32d1-417b-9427-13669ba6ac52 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.353697 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64213f47-32d1-417b-9427-13669ba6ac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.353990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64213f47-32d1-417b-9427-13669ba6ac52 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.354607 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64213f47-32d1-417b-9427-13669ba6ac52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:44.356482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64213f47-32d1-417b-9427-13669ba6ac52 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:44.444597 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1274/2510] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:50:44 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:50:44.448573 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567adaf-1b5d-4d82-a018-b498845cafc5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:44.459441 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567adaf-1b5d-4d82-a018-b498845cafc5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:44.464087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730154a0-9e25-4178-b511-c233e78f08f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:44.464672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730154a0-9e25-4178-b511-c233e78f08f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.464949 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730154a0-9e25-4178-b511-c233e78f08f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.465221 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730154a0-9e25-4178-b511-c233e78f08f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.465850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730154a0-9e25-4178-b511-c233e78f08f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:44.467544 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730154a0-9e25-4178-b511-c233e78f08f7 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:44.477373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1237/2511] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:44.481161 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64213f47-32d1-417b-9427-13669ba6ac52 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:44.491008 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64213f47-32d1-417b-9427-13669ba6ac52 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:44.495515 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce58468-07e3-4a54-8671-fa22dce5040e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:44.496095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce58468-07e3-4a54-8671-fa22dce5040e admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.496373 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce58468-07e3-4a54-8671-fa22dce5040e admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.496642 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce58468-07e3-4a54-8671-fa22dce5040e admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.497272 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce58468-07e3-4a54-8671-fa22dce5040e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:44.498860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce58468-07e3-4a54-8671-fa22dce5040e admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:44.510025 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1275/2512] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:50:44.514170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730154a0-9e25-4178-b511-c233e78f08f7 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:44.524586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730154a0-9e25-4178-b511-c233e78f08f7 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:44.529066 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:44.529695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.530003 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.530276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] PATH_INFO: `/v3/projects/3f26b25757db4547839cab3d1108f21a/users/1e6cc16d9a8241bc90ca1ae054508dbf/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.544928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f26b25757db4547839cab3d1108f21a, user_id=1e6cc16d9a8241bc90ca1ae054508dbf, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:44.547610 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:44.635807 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1238/2513] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:50:44 2022] PUT /identity/v3/projects/3f26b25757db4547839cab3d1108f21a/users/1e6cc16d9a8241bc90ca1ae054508dbf/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:44.647367 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227fa4b-f074-491a-869c-05ecfcfb5fba None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.647652 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227fa4b-f074-491a-869c-05ecfcfb5fba None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.647926 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227fa4b-f074-491a-869c-05ecfcfb5fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.675943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-1227fa4b-f074-491a-869c-05ecfcfb5fba None None] MFA Rules not processed for user `1e6cc16d9a8241bc90ca1ae054508dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:44.677187 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-1227fa4b-f074-491a-869c-05ecfcfb5fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:44.740215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1276/2514] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:50:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:44.751793 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:44.754193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:44.831313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565cf7d-ce00-4e88-a9b3-9f04c05e5b5b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:44.834883 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b7924-fdcf-4b58-ab99-7c288590da52 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cbcaeb843e470781661f00ec91bffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:44.835475 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b7924-fdcf-4b58-ab99-7c288590da52 service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:44.835750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b7924-fdcf-4b58-ab99-7c288590da52 service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:44.836019 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b7924-fdcf-4b58-ab99-7c288590da52 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:44.844187 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b7924-fdcf-4b58-ab99-7c288590da52 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:44.845589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b7924-fdcf-4b58-ab99-7c288590da52 service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:44.854262 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1239/2515] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:54.736801 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:54.850316 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:02.041028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227fa4b-f074-491a-869c-05ecfcfb5fba None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:02.126349 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227fa4b-f074-491a-869c-05ecfcfb5fba None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:02.130972 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:02.131865 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:02.132148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:02.132417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] PATH_INFO: `/v3/users/1e6cc16d9a8241bc90ca1ae054508dbf` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:02.133559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6cc16d9a8241bc90ca1ae054508dbf)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:02.135573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:02.291268 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6cc16d9a8241bc90ca1ae054508dbf'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:02.305048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6cc16d9a8241bc90ca1ae054508dbf'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:02.305747 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6cc16d9a8241bc90ca1ae054508dbf'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:02.363281 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6cc16d9a8241bc90ca1ae054508dbf'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:02.372203 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6cc16d9a8241bc90ca1ae054508dbf'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:02.378491 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1277/2516] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:51:02 2022] DELETE /identity/v3/users/1e6cc16d9a8241bc90ca1ae054508dbf => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:02.383517 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86f43a-55f0-45bb-a7d4-a4c462c3cac1 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:02.384343 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86f43a-55f0-45bb-a7d4-a4c462c3cac1 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:02.384619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86f43a-55f0-45bb-a7d4-a4c462c3cac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:02.404342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4c86f43a-55f0-45bb-a7d4-a4c462c3cac1 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:02.405549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4c86f43a-55f0-45bb-a7d4-a4c462c3cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:02.467304 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1240/2517] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:02.477209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:02.478713 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:02.538561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871b572-a3a9-460b-9342-112a22a991ce admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:02.543122 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827feb9-20b2-418d-bb9b-55f0a8154ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:02.543669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827feb9-20b2-418d-bb9b-55f0a8154ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:02.543953 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827feb9-20b2-418d-bb9b-55f0a8154ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:02.544226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827feb9-20b2-418d-bb9b-55f0a8154ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:02.554819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827feb9-20b2-418d-bb9b-55f0a8154ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:02.556251 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827feb9-20b2-418d-bb9b-55f0a8154ea3 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:02.566857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1278/2518] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:02.968952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86f43a-55f0-45bb-a7d4-a4c462c3cac1 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:02.968952 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4c86f43a-55f0-45bb-a7d4-a4c462c3cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:02.978150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86f43a-55f0-45bb-a7d4-a4c462c3cac1 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:02.983136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:02.984028 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:02.984308 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:02.984589 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] PATH_INFO: `/v3/projects/3f26b25757db4547839cab3d1108f21a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:02.985736 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f26b25757db4547839cab3d1108f21a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:02.987839 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:03.100369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f26b25757db4547839cab3d1108f21a', 'request_id': 'req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:03.105774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f26b25757db4547839cab3d1108f21a', 'request_id': 'req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:03.112667 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f26b25757db4547839cab3d1108f21a', 'request_id': 'req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:03.182481 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] The token cache is being invalidate because project 3f26b25757db4547839cab3d1108f21a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:51:03.182850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:03.183486 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-ae1f7f21-ad8c-45eb-8c2e-477002a887e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:03.184989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1241/2519] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:51:02 2022] DELETE /identity/v3/projects/3f26b25757db4547839cab3d1108f21a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:03.189851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c695d-9b14-435a-995d-877a964c84f1 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.190328 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c695d-9b14-435a-995d-877a964c84f1 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.190435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c695d-9b14-435a-995d-877a964c84f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.211541 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-9f2c695d-9b14-435a-995d-877a964c84f1 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:03.212741 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-9f2c695d-9b14-435a-995d-877a964c84f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:03.230983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:03.274050 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1279/2520] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:03.281917 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d9efb-553f-440e-bd1d-6d997c04ef11 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.282304 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d9efb-553f-440e-bd1d-6d997c04ef11 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.282583 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d9efb-553f-440e-bd1d-6d997c04ef11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.303863 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-4b2d9efb-553f-440e-bd1d-6d997c04ef11 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:03.305084 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-4b2d9efb-553f-440e-bd1d-6d997c04ef11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:03.314027 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1242/2521] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:03.318026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c695d-9b14-435a-995d-877a964c84f1 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:03.330540 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c695d-9b14-435a-995d-877a964c84f1 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:03.333621 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:03.334266 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.334542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.334808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] PATH_INFO: `/v3/domains` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.335516 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:51:03.335819 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:03.337398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:03.356441 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1280/2522] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:51:03.359686 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d9efb-553f-440e-bd1d-6d997c04ef11 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:03.369497 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d9efb-553f-440e-bd1d-6d997c04ef11 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:03.374353 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be7f9-ec0e-4927-b0d4-58ac22aebc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:03.374946 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be7f9-ec0e-4927-b0d4-58ac22aebc30 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.375226 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be7f9-ec0e-4927-b0d4-58ac22aebc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.375498 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be7f9-ec0e-4927-b0d4-58ac22aebc30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.376136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be7f9-ec0e-4927-b0d4-58ac22aebc30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:03.378036 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be7f9-ec0e-4927-b0d4-58ac22aebc30 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:03.462866 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1243/2523] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:51:03 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:51:03.467033 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:03.477093 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67eaf0-0f2d-462c-9c15-57bdc14ae34b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:03.534165 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0891d9-abb1-45a7-ae48-20888dab11b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:03.534758 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0891d9-abb1-45a7-ae48-20888dab11b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.535036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0891d9-abb1-45a7-ae48-20888dab11b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.535307 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0891d9-abb1-45a7-ae48-20888dab11b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.535929 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0891d9-abb1-45a7-ae48-20888dab11b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:03.537839 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0891d9-abb1-45a7-ae48-20888dab11b1 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:03.633658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1281/2524] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:51:03 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:51:03.638176 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be7f9-ec0e-4927-b0d4-58ac22aebc30 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:03.648425 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be7f9-ec0e-4927-b0d4-58ac22aebc30 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:03.653150 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d3254-1052-4b33-952c-ea3b53f321f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:03.653741 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d3254-1052-4b33-952c-ea3b53f321f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.654055 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d3254-1052-4b33-952c-ea3b53f321f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.654329 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d3254-1052-4b33-952c-ea3b53f321f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.655040 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d3254-1052-4b33-952c-ea3b53f321f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:03.656551 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d3254-1052-4b33-952c-ea3b53f321f3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:03.667413 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1244/2525] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:51:03.671504 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0891d9-abb1-45a7-ae48-20888dab11b1 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:03.682435 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0891d9-abb1-45a7-ae48-20888dab11b1 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:03.687262 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac99e8-7f5b-4dee-9ca7-1f44ac4e876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:03.697385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac99e8-7f5b-4dee-9ca7-1f44ac4e876b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.697385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac99e8-7f5b-4dee-9ca7-1f44ac4e876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.697385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac99e8-7f5b-4dee-9ca7-1f44ac4e876b admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.697385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac99e8-7f5b-4dee-9ca7-1f44ac4e876b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:03.697385 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac99e8-7f5b-4dee-9ca7-1f44ac4e876b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:03.702058 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1282/2526] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:51:03.706400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d3254-1052-4b33-952c-ea3b53f321f3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:03.716472 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d3254-1052-4b33-952c-ea3b53f321f3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:03.721203 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7522-65da-4ccf-9ff7-9ab00c719612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:03.721850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7522-65da-4ccf-9ff7-9ab00c719612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.722163 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7522-65da-4ccf-9ff7-9ab00c719612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.722442 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7522-65da-4ccf-9ff7-9ab00c719612 admin admin] PATH_INFO: `/v3/projects/2aafe4250425416eb6b438d064478015/users/37e9e1dc9980499abe360fd73cc68c36/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.739169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7522-65da-4ccf-9ff7-9ab00c719612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aafe4250425416eb6b438d064478015, user_id=37e9e1dc9980499abe360fd73cc68c36, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:03.741828 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7522-65da-4ccf-9ff7-9ab00c719612 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:03.814264 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1245/2527] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:51:03 2022] PUT /identity/v3/projects/2aafe4250425416eb6b438d064478015/users/37e9e1dc9980499abe360fd73cc68c36/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:03.825616 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9a03b-2be9-4c27-a352-bd1af4a8848a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.825905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9a03b-2be9-4c27-a352-bd1af4a8848a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.826209 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9a03b-2be9-4c27-a352-bd1af4a8848a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.855101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-55e9a03b-2be9-4c27-a352-bd1af4a8848a None None] MFA Rules not processed for user `37e9e1dc9980499abe360fd73cc68c36`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:03.856442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-55e9a03b-2be9-4c27-a352-bd1af4a8848a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:03.921860 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1283/2528] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:03.926639 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7522-65da-4ccf-9ff7-9ab00c719612 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:03.947713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:03.988449 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7522-65da-4ccf-9ff7-9ab00c719612 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:03.994462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60d9e6-ffb9-4fbf-9d8f-cfed6be5fab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:03.995360 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d9e6-ffb9-4fbf-9d8f-cfed6be5fab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:03.995637 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d9e6-ffb9-4fbf-9d8f-cfed6be5fab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:03.995908 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d9e6-ffb9-4fbf-9d8f-cfed6be5fab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:03.996533 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60d9e6-ffb9-4fbf-9d8f-cfed6be5fab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:03.998449 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60d9e6-ffb9-4fbf-9d8f-cfed6be5fab4 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:04.117897 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1246/2529] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:51:03 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:51:04.119601 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9a03b-2be9-4c27-a352-bd1af4a8848a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:04.130585 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9a03b-2be9-4c27-a352-bd1af4a8848a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:04.146717 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:04.195754 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355eaab-37d3-457e-adab-a7e45823f41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:04.196419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355eaab-37d3-457e-adab-a7e45823f41f admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:04.197320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355eaab-37d3-457e-adab-a7e45823f41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:04.197592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355eaab-37d3-457e-adab-a7e45823f41f admin admin] PATH_INFO: `/v3/users` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:04.198303 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355eaab-37d3-457e-adab-a7e45823f41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:04.200243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355eaab-37d3-457e-adab-a7e45823f41f admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:04.273718 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1284/2530] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:51:04 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:51:04.277663 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60d9e6-ffb9-4fbf-9d8f-cfed6be5fab4 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:04.289785 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60d9e6-ffb9-4fbf-9d8f-cfed6be5fab4 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:04.294762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8d41c-fa8c-4bd0-af8b-910b6c458e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:04.295438 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8d41c-fa8c-4bd0-af8b-910b6c458e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:04.295776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8d41c-fa8c-4bd0-af8b-910b6c458e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:04.296023 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8d41c-fa8c-4bd0-af8b-910b6c458e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:04.296772 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8d41c-fa8c-4bd0-af8b-910b6c458e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:04.298652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8d41c-fa8c-4bd0-af8b-910b6c458e42 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:04.310196 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1247/2531] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:51:04.315178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355eaab-37d3-457e-adab-a7e45823f41f admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:04.327908 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355eaab-37d3-457e-adab-a7e45823f41f admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:04.334372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f2c0e-8c3d-47dc-a0bb-44b22a47bdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:04.335306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f2c0e-8c3d-47dc-a0bb-44b22a47bdb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:04.335670 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f2c0e-8c3d-47dc-a0bb-44b22a47bdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:04.336140 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f2c0e-8c3d-47dc-a0bb-44b22a47bdb7 admin admin] PATH_INFO: `/v3/projects/808f865e1fad4cd388196c61b18442cc/users/1d572a5ab4cb4e548a6a0b0b0f1acbe1/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:04.355952 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f2c0e-8c3d-47dc-a0bb-44b22a47bdb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808f865e1fad4cd388196c61b18442cc, user_id=1d572a5ab4cb4e548a6a0b0b0f1acbe1, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:04.359313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f2c0e-8c3d-47dc-a0bb-44b22a47bdb7 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:04.483089 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1285/2532] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:51:04 2022] PUT /identity/v3/projects/808f865e1fad4cd388196c61b18442cc/users/1d572a5ab4cb4e548a6a0b0b0f1acbe1/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:04.484652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8d41c-fa8c-4bd0-af8b-910b6c458e42 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:04.495876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8d41c-fa8c-4bd0-af8b-910b6c458e42 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:04.554622 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1775c-3436-4202-99c6-0e1c05532014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:04.555252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1775c-3436-4202-99c6-0e1c05532014 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:04.555519 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1775c-3436-4202-99c6-0e1c05532014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:04.556605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1775c-3436-4202-99c6-0e1c05532014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:04.557258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1775c-3436-4202-99c6-0e1c05532014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:04.558888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1775c-3436-4202-99c6-0e1c05532014 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:04.569994 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1248/2533] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:51:04.573970 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f2c0e-8c3d-47dc-a0bb-44b22a47bdb7 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:04.584622 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f2c0e-8c3d-47dc-a0bb-44b22a47bdb7 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:04.590208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:04.590957 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:04.591330 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:04.591673 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] PATH_INFO: `/v3/projects/808f865e1fad4cd388196c61b18442cc/users/1d572a5ab4cb4e548a6a0b0b0f1acbe1/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:04.593662 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=808f865e1fad4cd388196c61b18442cc, user_id=1d572a5ab4cb4e548a6a0b0b0f1acbe1, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:04.596419 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:04.609414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1286/2534] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:51:04 2022] PUT /identity/v3/projects/808f865e1fad4cd388196c61b18442cc/users/1d572a5ab4cb4e548a6a0b0b0f1acbe1/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:04.620400 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3138c42-b8b9-49ba-9e07-25cb3163a68e None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:04.620687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3138c42-b8b9-49ba-9e07-25cb3163a68e None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:04.620966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3138c42-b8b9-49ba-9e07-25cb3163a68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:04.653193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-b3138c42-b8b9-49ba-9e07-25cb3163a68e None None] MFA Rules not processed for user `1d572a5ab4cb4e548a6a0b0b0f1acbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:04.654460 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b3138c42-b8b9-49ba-9e07-25cb3163a68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:04.723588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1249/2535] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:04.735094 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:04.736762 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:04.796640 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395d206-9146-4c74-aafb-2de566820b3b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:04.800405 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf32bb-664c-416a-846d-62812a7ddb67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:04.800996 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf32bb-664c-416a-846d-62812a7ddb67 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:04.801290 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf32bb-664c-416a-846d-62812a7ddb67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:04.801544 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf32bb-664c-416a-846d-62812a7ddb67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:04.857121 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf32bb-664c-416a-846d-62812a7ddb67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:04.858588 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf32bb-664c-416a-846d-62812a7ddb67 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:04.868221 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1287/2536] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:51:07.951569 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3138c42-b8b9-49ba-9e07-25cb3163a68e None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:07.969134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:08.015500 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3138c42-b8b9-49ba-9e07-25cb3163a68e None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:08.023242 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:08.024197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:08.024197 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:08.024395 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] PATH_INFO: `/v3/users/37e9e1dc9980499abe360fd73cc68c36` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:08.025469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e9e1dc9980499abe360fd73cc68c36)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:08.027484 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:08.129063 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e9e1dc9980499abe360fd73cc68c36'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:08.141352 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e9e1dc9980499abe360fd73cc68c36'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:08.212895 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e9e1dc9980499abe360fd73cc68c36'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:08.213548 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e9e1dc9980499abe360fd73cc68c36'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:08.220687 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e9e1dc9980499abe360fd73cc68c36'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:08.226526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1250/2537] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:51:07 2022] DELETE /identity/v3/users/37e9e1dc9980499abe360fd73cc68c36 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:08.231184 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77af6-ad25-4ab5-8b8b-8fec2b831c40 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:08.231658 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77af6-ad25-4ab5-8b8b-8fec2b831c40 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:08.231768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77af6-ad25-4ab5-8b8b-8fec2b831c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:08.250592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-d5e77af6-ad25-4ab5-8b8b-8fec2b831c40 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:08.251756 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d5e77af6-ad25-4ab5-8b8b-8fec2b831c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:08.270525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:08.310647 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1288/2538] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:08.320860 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:08.322341 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:08.379050 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a0777-0891-4d52-b9f2-95ec98cd705e admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:08.383259 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fe96d-b4ee-454c-8aba-f49b99b9be2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:08.383972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fe96d-b4ee-454c-8aba-f49b99b9be2e service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:08.384258 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fe96d-b4ee-454c-8aba-f49b99b9be2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:08.384532 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fe96d-b4ee-454c-8aba-f49b99b9be2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:08.393601 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fe96d-b4ee-454c-8aba-f49b99b9be2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:08.395080 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fe96d-b4ee-454c-8aba-f49b99b9be2e service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:08.405263 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1251/2539] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:08.791250 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e77af6-ad25-4ab5-8b8b-8fec2b831c40 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:08.792847 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-d5e77af6-ad25-4ab5-8b8b-8fec2b831c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:08.807473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e77af6-ad25-4ab5-8b8b-8fec2b831c40 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:08.812130 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:08.812761 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:08.813038 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:08.813306 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] PATH_INFO: `/v3/projects/2aafe4250425416eb6b438d064478015` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:08.814442 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aafe4250425416eb6b438d064478015)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:08.816335 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:08.921594 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aafe4250425416eb6b438d064478015', 'request_id': 'req-ec84b0e0-47a1-460e-9042-7bd23b1c1110'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:08.927084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aafe4250425416eb6b438d064478015', 'request_id': 'req-ec84b0e0-47a1-460e-9042-7bd23b1c1110'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:08.933875 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aafe4250425416eb6b438d064478015', 'request_id': 'req-ec84b0e0-47a1-460e-9042-7bd23b1c1110'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.005648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] The token cache is being invalidate because project 2aafe4250425416eb6b438d064478015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:51:09.005958 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.006579 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.008084 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1289/2540] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:51:08 2022] DELETE /identity/v3/projects/2aafe4250425416eb6b438d064478015 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:09.012057 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fe96d-b4ee-454c-8aba-f49b99b9be2e service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:09.021923 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fe96d-b4ee-454c-8aba-f49b99b9be2e service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:09.057335 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:09.081373 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:09.082034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:09.082321 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:09.082596 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] PATH_INFO: `/v3/users/1d572a5ab4cb4e548a6a0b0b0f1acbe1` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:09.083722 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d572a5ab4cb4e548a6a0b0b0f1acbe1)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:09.085670 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:09.255275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d572a5ab4cb4e548a6a0b0b0f1acbe1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.267838 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d572a5ab4cb4e548a6a0b0b0f1acbe1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.315083 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d572a5ab4cb4e548a6a0b0b0f1acbe1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.315631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d572a5ab4cb4e548a6a0b0b0f1acbe1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.323479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-854e3491-c751-43ac-b9be-22ab9b20e3c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d572a5ab4cb4e548a6a0b0b0f1acbe1'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.329763 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1252/2541] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:51:09 2022] DELETE /identity/v3/users/1d572a5ab4cb4e548a6a0b0b0f1acbe1 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:09.728215 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:09.730085 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:09.744465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b0e0-47a1-460e-9042-7bd23b1c1110 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:09.807313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:09.810982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:09.810982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:09.810982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] PATH_INFO: `/v3/projects/808f865e1fad4cd388196c61b18442cc` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:09.810982 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=808f865e1fad4cd388196c61b18442cc)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:09.811466 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:09.925765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808f865e1fad4cd388196c61b18442cc', 'request_id': 'req-cc9c5206-4b05-455f-b466-bb41f7212717'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.927227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:09.932362 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808f865e1fad4cd388196c61b18442cc', 'request_id': 'req-cc9c5206-4b05-455f-b466-bb41f7212717'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:09.939470 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808f865e1fad4cd388196c61b18442cc', 'request_id': 'req-cc9c5206-4b05-455f-b466-bb41f7212717'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:10.024837 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] The token cache is being invalidate because project 808f865e1fad4cd388196c61b18442cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:51:10.025344 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:10.026179 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-cc9c5206-4b05-455f-b466-bb41f7212717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:10.028111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1290/2542] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:51:09 2022] DELETE /identity/v3/projects/808f865e1fad4cd388196c61b18442cc => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:51:10.032665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364732-0db5-4d18-82d9-324c49d40b6d None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.033123 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364732-0db5-4d18-82d9-324c49d40b6d None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.033251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f364732-0db5-4d18-82d9-324c49d40b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.053833 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6f364732-0db5-4d18-82d9-324c49d40b6d None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:10.055073 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6f364732-0db5-4d18-82d9-324c49d40b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:10.127600 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1253/2543] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:10.135948 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011094a3-3cd9-4c9f-a965-a4a17e822bd9 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.136301 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011094a3-3cd9-4c9f-a965-a4a17e822bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.136681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011094a3-3cd9-4c9f-a965-a4a17e822bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.158036 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-011094a3-3cd9-4c9f-a965-a4a17e822bd9 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:10.159521 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-011094a3-3cd9-4c9f-a965-a4a17e822bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:10.168645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1291/2544] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:10.172905 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f364732-0db5-4d18-82d9-324c49d40b6d None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:10.184541 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f364732-0db5-4d18-82d9-324c49d40b6d None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:10.190327 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:10.190966 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.191238 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.191534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.192936 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:51:10.193074 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:10.194705 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:10.207779 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1254/2545] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:51:10.212712 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011094a3-3cd9-4c9f-a965-a4a17e822bd9 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:10.223095 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011094a3-3cd9-4c9f-a965-a4a17e822bd9 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:10.228489 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fb7f6-605c-42e9-bd19-b74ea710bbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:10.228753 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb7f6-605c-42e9-bd19-b74ea710bbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.229143 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb7f6-605c-42e9-bd19-b74ea710bbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.229479 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fb7f6-605c-42e9-bd19-b74ea710bbd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.230210 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fb7f6-605c-42e9-bd19-b74ea710bbd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:10.232174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fb7f6-605c-42e9-bd19-b74ea710bbd8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:10.329028 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1292/2546] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:51:10 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:51:10.329859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:10.343887 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7268a6-6728-4739-996f-3975c8785884 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:10.395924 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:10.412075 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef9858-8a25-44bb-b1f7-b2c3312460b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:10.412692 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef9858-8a25-44bb-b1f7-b2c3312460b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.413004 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef9858-8a25-44bb-b1f7-b2c3312460b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.413232 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef9858-8a25-44bb-b1f7-b2c3312460b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.413878 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef9858-8a25-44bb-b1f7-b2c3312460b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:10.415814 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef9858-8a25-44bb-b1f7-b2c3312460b3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:10.496120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1255/2547] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:51:10 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:51:10.500162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fb7f6-605c-42e9-bd19-b74ea710bbd8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:10.512305 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fb7f6-605c-42e9-bd19-b74ea710bbd8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:10.534237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbee47-b27a-4fa3-9997-335e142c9b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:10.534237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbee47-b27a-4fa3-9997-335e142c9b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.534237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbee47-b27a-4fa3-9997-335e142c9b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.534237 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbee47-b27a-4fa3-9997-335e142c9b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.534648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbee47-b27a-4fa3-9997-335e142c9b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:10.534648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbee47-b27a-4fa3-9997-335e142c9b38 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:10.534648 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1293/2548] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:51:10.535219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef9858-8a25-44bb-b1f7-b2c3312460b3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:10.546523 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef9858-8a25-44bb-b1f7-b2c3312460b3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:10.552100 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85006b13-8791-458c-a784-6ea07822dd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:10.552748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85006b13-8791-458c-a784-6ea07822dd35 admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.552999 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85006b13-8791-458c-a784-6ea07822dd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.553272 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85006b13-8791-458c-a784-6ea07822dd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.553916 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85006b13-8791-458c-a784-6ea07822dd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:10.555564 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85006b13-8791-458c-a784-6ea07822dd35 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:10.570676 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1256/2549] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:51:10.571766 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbee47-b27a-4fa3-9997-335e142c9b38 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:10.581877 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbee47-b27a-4fa3-9997-335e142c9b38 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:10.586750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:10.587394 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.587669 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.587939 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] PATH_INFO: `/v3/projects/172681aae5d649b7b8cc5dd50779a631/users/ed1e915d562a466cacb53942e1198333/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.603637 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] RBAC: Authorizing `identity:create_grant(project_id=172681aae5d649b7b8cc5dd50779a631, user_id=ed1e915d562a466cacb53942e1198333, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:10.606207 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:10.682611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1294/2550] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:51:10 2022] PUT /identity/v3/projects/172681aae5d649b7b8cc5dd50779a631/users/ed1e915d562a466cacb53942e1198333/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:51:10.692496 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27ba3f-adfc-4681-b5d8-07b1b88c6a30 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.692780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27ba3f-adfc-4681-b5d8-07b1b88c6a30 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.693062 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27ba3f-adfc-4681-b5d8-07b1b88c6a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.727284 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8a27ba3f-adfc-4681-b5d8-07b1b88c6a30 None None] MFA Rules not processed for user `ed1e915d562a466cacb53942e1198333`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:10.728559 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8a27ba3f-adfc-4681-b5d8-07b1b88c6a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:10.803275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1257/2551] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:10.811979 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:10.813891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:10.874823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd101dcb-8845-484e-b865-01b0ac8cb78b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:10.879076 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8238fbc3012844ea8b2b5ab13c8ea5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:10.879728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:10.880006 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:10.880269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:10.889269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:10.890612 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:10.900533 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1295/2552] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:12.636122 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8395c-d980-486d-be12-13cf3d814f2b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:12.636534 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8395c-d980-486d-be12-13cf3d814f2b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:12.636849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8395c-d980-486d-be12-13cf3d814f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:12.658889 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a6e8395c-d980-486d-be12-13cf3d814f2b None None] MFA Rules not processed for user `ed1e915d562a466cacb53942e1198333`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:12.660131 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a6e8395c-d980-486d-be12-13cf3d814f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:12.668190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1258/2553] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:16.326816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:16.328299 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:16.386779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33270e6b-3b44-456c-9456-d950dbf6dff7 service cinder] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:16.390418 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:16.391009 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:16.391930 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:16.392243 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:16.394048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:16.425154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:16.458909 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:16.460257 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:16.470623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1296/2554] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:18.363972 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8395c-d980-486d-be12-13cf3d814f2b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:18.366260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a6e8395c-d980-486d-be12-13cf3d814f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:18.447685 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8395c-d980-486d-be12-13cf3d814f2b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:18.454165 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:18.455025 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:18.455269 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:18.455646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:18.457576 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:18.470267 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:18.471462 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:18.484346 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1259/2555] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:20.903750 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:20.905356 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:20.921223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede207b-ac59-441b-b703-531a6426e51a service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:20.927635 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:20.928240 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:20.928755 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:20.929197 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:20.931034 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:20.944605 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:20.946379 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:20.958546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1297/2556] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:24.500189 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:24.501672 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:24.516137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f4dd8-e0d9-433e-a68e-ebbab8231e3e service placement] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:24.520646 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:24.521273 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:24.521549 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:24.526205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] PATH_INFO: `/v3/users/205b19ccb80a45fd8ed36abd2c4c32c3` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:24.526205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=205b19ccb80a45fd8ed36abd2c4c32c3)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:24.526205 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:24.684375 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:24.697979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205b19ccb80a45fd8ed36abd2c4c32c3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:24.710144 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205b19ccb80a45fd8ed36abd2c4c32c3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:24.794251 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205b19ccb80a45fd8ed36abd2c4c32c3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:24.794861 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205b19ccb80a45fd8ed36abd2c4c32c3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:24.802087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205b19ccb80a45fd8ed36abd2c4c32c3'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:24.808450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1260/2557] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:51:24 2022] DELETE /identity/v3/users/205b19ccb80a45fd8ed36abd2c4c32c3 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:25.047320 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:25.059457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3be5bb-1764-4ec3-82a0-3a12fc567d00 service placement] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:25.125068 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:25.125810 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:25.126297 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:25.126572 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] PATH_INFO: `/v3/projects/00c9aab9a0264449ac51194afd4cb6a9` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:25.127748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00c9aab9a0264449ac51194afd4cb6a9)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:25.129680 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:25.263736 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00c9aab9a0264449ac51194afd4cb6a9', 'request_id': 'req-60c0cca2-1af4-4e07-aff1-373187568c11'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.269286 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00c9aab9a0264449ac51194afd4cb6a9', 'request_id': 'req-60c0cca2-1af4-4e07-aff1-373187568c11'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.275915 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00c9aab9a0264449ac51194afd4cb6a9', 'request_id': 'req-60c0cca2-1af4-4e07-aff1-373187568c11'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.364771 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] The token cache is being invalidate because project 00c9aab9a0264449ac51194afd4cb6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:51:25.364981 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.365592 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.367390 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1298/2558] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:51:25 2022] DELETE /identity/v3/projects/00c9aab9a0264449ac51194afd4cb6a9 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:25.371766 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:25.373514 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:25.387614 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f6098-fc8e-485a-bb7b-3c2f13c12eed admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:25.448913 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:25.449553 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:25.449840 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:25.450169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] PATH_INFO: `/v3/users/f5c74c3045d6458ea02bf89e3d62d18a` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:25.451355 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c74c3045d6458ea02bf89e3d62d18a)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:25.453297 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:25.614195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c74c3045d6458ea02bf89e3d62d18a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.626832 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c74c3045d6458ea02bf89e3d62d18a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.696435 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c74c3045d6458ea02bf89e3d62d18a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.697229 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c74c3045d6458ea02bf89e3d62d18a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.705693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-7e4bf744-1e98-40c9-8359-13a22bbb6272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c74c3045d6458ea02bf89e3d62d18a'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:25.712016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1261/2559] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:51:25 2022] DELETE /identity/v3/users/f5c74c3045d6458ea02bf89e3d62d18a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:25.963213 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:25.965270 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:25.978559 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0cca2-1af4-4e07-aff1-373187568c11 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:26.022088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:26.037245 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:26.037873 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:26.038180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:26.038455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] PATH_INFO: `/v3/projects/59c7b59608464ffdbe9f70ad618aa044` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:26.039561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c7b59608464ffdbe9f70ad618aa044)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:26.041525 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:26.137274 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c7b59608464ffdbe9f70ad618aa044', 'request_id': 'req-5d844860-4456-4a0f-999b-75e1f2d10f70'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:26.142786 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c7b59608464ffdbe9f70ad618aa044', 'request_id': 'req-5d844860-4456-4a0f-999b-75e1f2d10f70'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:26.149265 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c7b59608464ffdbe9f70ad618aa044', 'request_id': 'req-5d844860-4456-4a0f-999b-75e1f2d10f70'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:26.217977 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] The token cache is being invalidate because project 59c7b59608464ffdbe9f70ad618aa044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:51:26.218218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:26.218852 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:26.220583 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1299/2560] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:51:25 2022] DELETE /identity/v3/projects/59c7b59608464ffdbe9f70ad618aa044 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:30.833957 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dcad0-02b2-4595-93a8-6503f09a8fdd None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:30.834470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dcad0-02b2-4595-93a8-6503f09a8fdd None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:30.834611 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dcad0-02b2-4595-93a8-6503f09a8fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:30.870666 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:30.893377 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-f98dcad0-02b2-4595-93a8-6503f09a8fdd None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:30.894602 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f98dcad0-02b2-4595-93a8-6503f09a8fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:30.957714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1262/2561] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:51:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:51:31.088313 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:31.089728 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:31.178781 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d844860-4456-4a0f-999b-75e1f2d10f70 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:31.184453 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f37b9-c559-43a7-9d35-9208b7dda79e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:31.185052 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f37b9-c559-43a7-9d35-9208b7dda79e service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:31.185355 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f37b9-c559-43a7-9d35-9208b7dda79e service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:31.185629 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f37b9-c559-43a7-9d35-9208b7dda79e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:31.194575 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f37b9-c559-43a7-9d35-9208b7dda79e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:31.196200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f37b9-c559-43a7-9d35-9208b7dda79e service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:31.204445 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1300/2562] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:51:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:51:31.958067 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d915-98f2-4929-987f-c445a2c79ea8 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:31.958563 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d915-98f2-4929-987f-c445a2c79ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:31.958665 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97d915-98f2-4929-987f-c445a2c79ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:31.977991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-8a97d915-98f2-4929-987f-c445a2c79ea8 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:31.979169 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-8a97d915-98f2-4929-987f-c445a2c79ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:31.986177 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1263/2563] 10.222.0.44 () {58 vars in 967 bytes} [Wed Oct 12 12:51:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:31.998340 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f37b9-c559-43a7-9d35-9208b7dda79e service swift] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:32.009447 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f37b9-c559-43a7-9d35-9208b7dda79e service swift] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:32.014573 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cb3ad-08f6-43ae-979d-4af6f1928987 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:32.015233 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cb3ad-08f6-43ae-979d-4af6f1928987 service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:32.015513 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cb3ad-08f6-43ae-979d-4af6f1928987 service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:32.015778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cb3ad-08f6-43ae-979d-4af6f1928987 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:32.024256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cb3ad-08f6-43ae-979d-4af6f1928987 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:32.025788 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cb3ad-08f6-43ae-979d-4af6f1928987 service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:32.033566 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1301/2564] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:51:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:51:32.614480 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ff822-c5de-40f6-8742-445fd38094b7 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:32.614876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ff822-c5de-40f6-8742-445fd38094b7 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:32.615148 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ff822-c5de-40f6-8742-445fd38094b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:32.634959 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-3c0ff822-c5de-40f6-8742-445fd38094b7 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:32.636134 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3c0ff822-c5de-40f6-8742-445fd38094b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:32.643313 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1264/2565] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:51:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:32.655004 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cb3ad-08f6-43ae-979d-4af6f1928987 service swift] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:32.665994 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cb3ad-08f6-43ae-979d-4af6f1928987 service swift] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:32.671019 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2d19b-fa33-4614-8576-4dc96f3e01ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:32.671606 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2d19b-fa33-4614-8576-4dc96f3e01ad service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:32.671887 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2d19b-fa33-4614-8576-4dc96f3e01ad service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:32.672157 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2d19b-fa33-4614-8576-4dc96f3e01ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:32.680902 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2d19b-fa33-4614-8576-4dc96f3e01ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:32.682496 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2d19b-fa33-4614-8576-4dc96f3e01ad service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:32.690791 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1302/2566] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:51:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:51:33.220585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a635c7-284a-48c2-bda0-3ccc91893641 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:33.221087 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a635c7-284a-48c2-bda0-3ccc91893641 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:33.221190 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a635c7-284a-48c2-bda0-3ccc91893641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:33.240125 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-a9a635c7-284a-48c2-bda0-3ccc91893641 None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:33.241296 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a9a635c7-284a-48c2-bda0-3ccc91893641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:33.248193 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1265/2567] 10.222.0.44 () {58 vars in 967 bytes} [Wed Oct 12 12:51:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:33.260620 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2d19b-fa33-4614-8576-4dc96f3e01ad service swift] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:33.271076 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2d19b-fa33-4614-8576-4dc96f3e01ad service swift] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:33.276650 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3505aca-5201-4f1c-901f-77a8bd6c5719 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:33.277226 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3505aca-5201-4f1c-901f-77a8bd6c5719 service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:33.277501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3505aca-5201-4f1c-901f-77a8bd6c5719 service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:33.277768 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3505aca-5201-4f1c-901f-77a8bd6c5719 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:33.285486 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3505aca-5201-4f1c-901f-77a8bd6c5719 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:33.287049 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3505aca-5201-4f1c-901f-77a8bd6c5719 service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:33.294879 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1303/2568] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:51:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:51:35.948809 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a635c7-284a-48c2-bda0-3ccc91893641 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:35.950216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-a9a635c7-284a-48c2-bda0-3ccc91893641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:36.013594 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a635c7-284a-48c2-bda0-3ccc91893641 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:36.018095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:36.018730 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:36.018989 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:36.019260 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] PATH_INFO: `/v3/users/ed1e915d562a466cacb53942e1198333` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:36.020368 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed1e915d562a466cacb53942e1198333)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:36.022294 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:36.232145 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed1e915d562a466cacb53942e1198333'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.244792 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed1e915d562a466cacb53942e1198333'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.315470 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed1e915d562a466cacb53942e1198333'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.316101 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed1e915d562a466cacb53942e1198333'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.317431 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:36.323738 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed1e915d562a466cacb53942e1198333'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.329542 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1266/2569] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:51:35 2022] DELETE /identity/v3/users/ed1e915d562a466cacb53942e1198333 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:36.334159 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718a0f9-3a8e-4ad1-b487-9450593e19e5 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:36.334639 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718a0f9-3a8e-4ad1-b487-9450593e19e5 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:36.334748 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718a0f9-3a8e-4ad1-b487-9450593e19e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:36.342561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:36.353978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5718a0f9-3a8e-4ad1-b487-9450593e19e5 None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:36.355180 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5718a0f9-3a8e-4ad1-b487-9450593e19e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:36.412428 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1304/2570] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:36.422714 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:36.424118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:36.481780 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11167c1-0937-4d79-a62a-71e9822380c3 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:36.485731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083a08e-d754-4e34-983a-647002b96d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:36.486306 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083a08e-d754-4e34-983a-647002b96d41 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:36.486586 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083a08e-d754-4e34-983a-647002b96d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:36.486859 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083a08e-d754-4e34-983a-647002b96d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:36.495888 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083a08e-d754-4e34-983a-647002b96d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:36.497369 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083a08e-d754-4e34-983a-647002b96d41 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:36.506819 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1267/2571] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:36.731962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718a0f9-3a8e-4ad1-b487-9450593e19e5 None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:36.733398 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5718a0f9-3a8e-4ad1-b487-9450593e19e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:36.748154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718a0f9-3a8e-4ad1-b487-9450593e19e5 None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:36.752943 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:36.756482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:36.756482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:36.756482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] PATH_INFO: `/v3/projects/172681aae5d649b7b8cc5dd50779a631` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:36.756482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] RBAC: Authorizing `identity:delete_project(project_id=172681aae5d649b7b8cc5dd50779a631)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:36.757535 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:36.884543 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172681aae5d649b7b8cc5dd50779a631', 'request_id': 'req-bfb9c7b8-9907-4811-bede-d44cb1a5361c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.890088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172681aae5d649b7b8cc5dd50779a631', 'request_id': 'req-bfb9c7b8-9907-4811-bede-d44cb1a5361c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.896729 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172681aae5d649b7b8cc5dd50779a631', 'request_id': 'req-bfb9c7b8-9907-4811-bede-d44cb1a5361c'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.969429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] The token cache is being invalidate because project 172681aae5d649b7b8cc5dd50779a631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:51:36.969733 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.970348 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-bfb9c7b8-9907-4811-bede-d44cb1a5361c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:51:36.971857 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1305/2572] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:51:36 2022] DELETE /identity/v3/projects/172681aae5d649b7b8cc5dd50779a631 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:36.977450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0cd48-7ba3-4261-840a-b9a334237f2a None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:36.977961 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0cd48-7ba3-4261-840a-b9a334237f2a None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:36.978095 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0cd48-7ba3-4261-840a-b9a334237f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:36.998752 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-bfd0cd48-7ba3-4261-840a-b9a334237f2a None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:36.999896 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-bfd0cd48-7ba3-4261-840a-b9a334237f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:37.058499 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1268/2573] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:51:37.066714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8c20c-c092-40d4-9693-7b469ef359dc None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.067010 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8c20c-c092-40d4-9693-7b469ef359dc None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.067501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8c20c-c092-40d4-9693-7b469ef359dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.087414 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-6ec8c20c-c092-40d4-9693-7b469ef359dc None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:37.088693 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-6ec8c20c-c092-40d4-9693-7b469ef359dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:37.096452 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1306/2574] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:37.101241 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0cd48-7ba3-4261-840a-b9a334237f2a None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:37.111276 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0cd48-7ba3-4261-840a-b9a334237f2a None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:37.115820 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:37.116412 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.117220 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.117494 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] PATH_INFO: `/v3/domains` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.118173 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:51:37.118479 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:37.120053 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:37.132979 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1269/2575] 10.222.0.44 () {58 vars in 1153 bytes} [Wed Oct 12 12:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:51:37.136990 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8c20c-c092-40d4-9693-7b469ef359dc None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:37.146255 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8c20c-c092-40d4-9693-7b469ef359dc None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:37.151357 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb6f65-e560-4187-a637-68c4f0eeb546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:37.151946 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb6f65-e560-4187-a637-68c4f0eeb546 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.152224 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb6f65-e560-4187-a637-68c4f0eeb546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.152490 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb6f65-e560-4187-a637-68c4f0eeb546 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.153103 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb6f65-e560-4187-a637-68c4f0eeb546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:37.155018 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fb6f65-e560-4187-a637-68c4f0eeb546 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:37.268458 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1307/2576] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:51:37 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:51:37.272605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:37.283180 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ceb8f-f49b-4022-96d8-bba05cd7b81b admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:37.340006 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17b91-1d0c-4a87-b364-3a0b9dd69e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:37.340605 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17b91-1d0c-4a87-b364-3a0b9dd69e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.340886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17b91-1d0c-4a87-b364-3a0b9dd69e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.341160 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17b91-1d0c-4a87-b364-3a0b9dd69e15 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.341783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d17b91-1d0c-4a87-b364-3a0b9dd69e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:37.343728 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d17b91-1d0c-4a87-b364-3a0b9dd69e15 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:37.426934 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1270/2577] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:51:37 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:51:37.431526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb6f65-e560-4187-a637-68c4f0eeb546 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:37.442053 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb6f65-e560-4187-a637-68c4f0eeb546 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:37.447276 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3181b-7f0e-4702-bcb3-c1739fd2a13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:37.452586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3181b-7f0e-4702-bcb3-c1739fd2a13b admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.452586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3181b-7f0e-4702-bcb3-c1739fd2a13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.452586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3181b-7f0e-4702-bcb3-c1739fd2a13b admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.452586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3181b-7f0e-4702-bcb3-c1739fd2a13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:37.452586 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3181b-7f0e-4702-bcb3-c1739fd2a13b admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:37.460802 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1308/2578] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:51:37.464631 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17b91-1d0c-4a87-b364-3a0b9dd69e15 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:37.474636 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17b91-1d0c-4a87-b364-3a0b9dd69e15 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:37.479309 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750ae2d-bef2-4eb0-8b0a-25c3c1bf188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:37.479937 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750ae2d-bef2-4eb0-8b0a-25c3c1bf188a admin admin] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.480216 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750ae2d-bef2-4eb0-8b0a-25c3c1bf188a admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.480488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750ae2d-bef2-4eb0-8b0a-25c3c1bf188a admin admin] PATH_INFO: `/v3/roles` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.481128 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750ae2d-bef2-4eb0-8b0a-25c3c1bf188a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:37.482750 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750ae2d-bef2-4eb0-8b0a-25c3c1bf188a admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:37.485068 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:37.494295 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1271/2579] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:51:37.502677 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3181b-7f0e-4702-bcb3-c1739fd2a13b admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:37.513169 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3181b-7f0e-4702-bcb3-c1739fd2a13b admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:37.518078 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:37.518542 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.518773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.519047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] PATH_INFO: `/v3/projects/7bc9dba7cc864f998bc613e10cf3b4aa/users/73179467b36a48a1b1c693dc9c026c9e/roles/92af8b532e69461b9088cc236a91aa39` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.533416 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc9dba7cc864f998bc613e10cf3b4aa, user_id=73179467b36a48a1b1c693dc9c026c9e, role_id=92af8b532e69461b9088cc236a91aa39)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:37.536088 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:37.604116 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1309/2580] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:51:37 2022] PUT /identity/v3/projects/7bc9dba7cc864f998bc613e10cf3b4aa/users/73179467b36a48a1b1c693dc9c026c9e/roles/92af8b532e69461b9088cc236a91aa39 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:37.609034 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9819-9c80-47e7-9da5-3d5c16bdd7af None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.609572 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9819-9c80-47e7-9da5-3d5c16bdd7af None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.609847 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9819-9c80-47e7-9da5-3d5c16bdd7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.629457 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-992e9819-9c80-47e7-9da5-3d5c16bdd7af None None] MFA Rules not processed for user `cd7cad0b678b4f0182359eae55570531`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:37.630682 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-992e9819-9c80-47e7-9da5-3d5c16bdd7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:37.690120 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1272/2581] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:37.700111 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:37.707200 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:37.730480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:37.759482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18c101-35df-4104-aea0-9359e1e571d8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:37.763371 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3285be-49b9-4a72-8289-66b48b10507d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:37.763969 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3285be-49b9-4a72-8289-66b48b10507d service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:37.764249 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3285be-49b9-4a72-8289-66b48b10507d service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:37.764520 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3285be-49b9-4a72-8289-66b48b10507d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:37.773654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3285be-49b9-4a72-8289-66b48b10507d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:37.775043 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3285be-49b9-4a72-8289-66b48b10507d service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:37.784851 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1310/2582] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:42.919783 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98451-26ee-4827-8e20-3f18be7797e6 None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:42.920293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98451-26ee-4827-8e20-3f18be7797e6 None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:42.920652 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98451-26ee-4827-8e20-3f18be7797e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:42.951821 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-30d98451-26ee-4827-8e20-3f18be7797e6 None None] MFA Rules not processed for user `73179467b36a48a1b1c693dc9c026c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:42.953030 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-30d98451-26ee-4827-8e20-3f18be7797e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:43.022526 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1273/2583] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:43.027279 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3285be-49b9-4a72-8289-66b48b10507d service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:43.037545 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3285be-49b9-4a72-8289-66b48b10507d service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:43.042212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dd83e-6846-46a0-b21a-42c6c1bbbe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:43.042814 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dd83e-6846-46a0-b21a-42c6c1bbbe67 admin admin] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:43.043092 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dd83e-6846-46a0-b21a-42c6c1bbbe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:43.043365 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dd83e-6846-46a0-b21a-42c6c1bbbe67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:43.044001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0dd83e-6846-46a0-b21a-42c6c1bbbe67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:43.045882 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0dd83e-6846-46a0-b21a-42c6c1bbbe67 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:43.160846 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1311/2584] 10.222.0.44 () {60 vars in 1153 bytes} [Wed Oct 12 12:51:43 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:51:43.162108 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d98451-26ee-4827-8e20-3f18be7797e6 None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:43.173467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d98451-26ee-4827-8e20-3f18be7797e6 None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:43.233527 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb9869-1524-4460-b39b-441ebc4714d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:43.234202 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9869-1524-4460-b39b-441ebc4714d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:43.234488 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9869-1524-4460-b39b-441ebc4714d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:43.234765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb9869-1524-4460-b39b-441ebc4714d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:43.239898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb9869-1524-4460-b39b-441ebc4714d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:43.239898 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb9869-1524-4460-b39b-441ebc4714d5 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:43.320765 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1274/2585] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Oct 12 12:51:43 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:51:43.324737 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dd83e-6846-46a0-b21a-42c6c1bbbe67 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:43.335449 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dd83e-6846-46a0-b21a-42c6c1bbbe67 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:43.340026 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaab7a1-cd09-48c3-84b3-f68aefdbcb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:43.340623 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaab7a1-cd09-48c3-84b3-f68aefdbcb1a admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:43.340906 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaab7a1-cd09-48c3-84b3-f68aefdbcb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:43.341178 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaab7a1-cd09-48c3-84b3-f68aefdbcb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:43.341817 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaab7a1-cd09-48c3-84b3-f68aefdbcb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:43.343465 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaab7a1-cd09-48c3-84b3-f68aefdbcb1a admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:43.354502 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1312/2586] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:51:43.358746 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb9869-1524-4460-b39b-441ebc4714d5 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:43.368850 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb9869-1524-4460-b39b-441ebc4714d5 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:43.373634 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72410ad-f37d-4bde-8b58-5d9b20d0c9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:43.374337 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72410ad-f37d-4bde-8b58-5d9b20d0c9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:43.374618 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72410ad-f37d-4bde-8b58-5d9b20d0c9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:43.374891 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72410ad-f37d-4bde-8b58-5d9b20d0c9c6 admin admin] PATH_INFO: `/v3/projects/91fbadf143bb4254bb64423fe3642cdd/users/7ddc94db34c84a51aa9bacc01972bf60/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:43.390565 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72410ad-f37d-4bde-8b58-5d9b20d0c9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fbadf143bb4254bb64423fe3642cdd, user_id=7ddc94db34c84a51aa9bacc01972bf60, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:43.393219 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72410ad-f37d-4bde-8b58-5d9b20d0c9c6 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:43.396225 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:43.489633 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1275/2587] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:51:43 2022] PUT /identity/v3/projects/91fbadf143bb4254bb64423fe3642cdd/users/7ddc94db34c84a51aa9bacc01972bf60/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:43.493772 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaab7a1-cd09-48c3-84b3-f68aefdbcb1a admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:43.504381 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaab7a1-cd09-48c3-84b3-f68aefdbcb1a admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:43.562558 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5f695-8183-4cf4-90fc-8c34843ff9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:43.563154 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5f695-8183-4cf4-90fc-8c34843ff9ec admin admin] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:43.563429 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5f695-8183-4cf4-90fc-8c34843ff9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:43.563695 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5f695-8183-4cf4-90fc-8c34843ff9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:43.564321 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5f695-8183-4cf4-90fc-8c34843ff9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:43.565891 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5f695-8183-4cf4-90fc-8c34843ff9ec admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:43.575599 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1313/2588] 10.222.0.44 () {58 vars in 1122 bytes} [Wed Oct 12 12:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:51:43.579713 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72410ad-f37d-4bde-8b58-5d9b20d0c9c6 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:43.590774 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72410ad-f37d-4bde-8b58-5d9b20d0c9c6 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:43.595853 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:43.596501 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] REQUEST_METHOD: `PUT` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:43.596778 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:43.597047 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] PATH_INFO: `/v3/projects/91fbadf143bb4254bb64423fe3642cdd/users/7ddc94db34c84a51aa9bacc01972bf60/roles/7b4139f251c84cfdadc3cfd82f0f81c0` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:43.599708 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fbadf143bb4254bb64423fe3642cdd, user_id=7ddc94db34c84a51aa9bacc01972bf60, role_id=7b4139f251c84cfdadc3cfd82f0f81c0)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:43.602829 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:43.613733 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1276/2589] 10.222.0.44 () {60 vars in 1483 bytes} [Wed Oct 12 12:51:43 2022] PUT /identity/v3/projects/91fbadf143bb4254bb64423fe3642cdd/users/7ddc94db34c84a51aa9bacc01972bf60/roles/7b4139f251c84cfdadc3cfd82f0f81c0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:51:49.132119 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02509379-ac2d-4941-89a9-ec87ec1a8419 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:49.132645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02509379-ac2d-4941-89a9-ec87ec1a8419 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:49.132645 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02509379-ac2d-4941-89a9-ec87ec1a8419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:49.163798 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-02509379-ac2d-4941-89a9-ec87ec1a8419 None None] MFA Rules not processed for user `7ddc94db34c84a51aa9bacc01972bf60`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:49.166148 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-02509379-ac2d-4941-89a9-ec87ec1a8419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:49.239795 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1314/2590] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:49.252991 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:49.255026 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:49.315886 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170f137-1d4a-4ce6-a07b-ed928c6d9727 admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:49.319613 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70cbcaeb843e470781661f00ec91bffd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:49.320223 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:49.320504 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:49.320776 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:49.379146 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:49.380585 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:49.390031 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1277/2591] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:51:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:51:49.654839 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fb55d-921f-48a4-ad63-d8864aa52a8e None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:49.655128 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fb55d-921f-48a4-ad63-d8864aa52a8e None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:49.655399 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fb55d-921f-48a4-ad63-d8864aa52a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:49.676482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-2c8fb55d-921f-48a4-ad63-d8864aa52a8e None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:49.677691 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-2c8fb55d-921f-48a4-ad63-d8864aa52a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:49.687214 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:49.732742 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1315/2592] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:51:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:51:49.745426 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:51:49.746823 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:49.816467 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7faa6-d62e-4ee1-881c-f6de914ecfb8 service glance] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:51:49.821437 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:51:49.822016 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:49.822299 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:49.822570 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:49.830707 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:51:49.832293 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:51:49.840485 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1278/2593] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:51:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:51:59.837140 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:INSERT=2,keystone:SELECT=87 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:51:59.852174 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66f5e9-8072-46bc-b3c3-efd29cb5d573 None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:51:59.852654 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66f5e9-8072-46bc-b3c3-efd29cb5d573 None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:51:59.852778 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66f5e9-8072-46bc-b3c3-efd29cb5d573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:51:59.871501 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-5b66f5e9-8072-46bc-b3c3-efd29cb5d573 None None] MFA Rules not processed for user `73179467b36a48a1b1c693dc9c026c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:51:59.872688 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-5b66f5e9-8072-46bc-b3c3-efd29cb5d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:51:59.879455 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1316/2594] 10.222.0.44 () {58 vars in 958 bytes} [Wed Oct 12 12:51:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:52:00.134689 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:52:00.136761 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:52:00.193137 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301d33-4475-45bf-b051-2eaec8a18e39 service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:52:00.196695 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:52:00.197291 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:52:00.197574 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:52:00.197849 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:52:00.200200 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:52:00.209656 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:52:00.211113 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:52:00.219136 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1279/2595] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:52:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:52:03.720473 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84946-881f-4af7-bedb-4986be7a8b9a None None] REQUEST_METHOD: `POST` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:52:03.721217 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84946-881f-4af7-bedb-4986be7a8b9a None None] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:52:03.721337 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe84946-881f-4af7-bedb-4986be7a8b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:52:03.740147 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.auth.core [None req-dbe84946-881f-4af7-bedb-4986be7a8b9a None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:52:03.741342 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-dbe84946-881f-4af7-bedb-4986be7a8b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:52:03.748546 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1317/2596] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Oct 12 12:52:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:52:03.761691 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:52:03.773693 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876237e3-544d-4344-a8c9-03fc11019c6a service neutron] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:52:03.779788 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d51e8-48c9-4715-a65b-f4e0dfba8fd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:52:03.780380 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d51e8-48c9-4715-a65b-f4e0dfba8fd8 service swift] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:52:03.780655 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d51e8-48c9-4715-a65b-f4e0dfba8fd8 service swift] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:52:03.780928 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d51e8-48c9-4715-a65b-f4e0dfba8fd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:52:03.789093 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d51e8-48c9-4715-a65b-f4e0dfba8fd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:52:03.790710 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d51e8-48c9-4715-a65b-f4e0dfba8fd8 service swift] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:52:03.798915 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1280/2597] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:52:13.736345 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=24 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:52:13.795995 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=32 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:52:31.627485 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe84946-881f-4af7-bedb-4986be7a8b9a None None] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:52:31.636816 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe84946-881f-4af7-bedb-4986be7a8b9a None None] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:52:31.640347 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2af8b-e8c9-44a7-855e-2d5d1a7db9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:52:31.640931 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2af8b-e8c9-44a7-855e-2d5d1a7db9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:52:31.641212 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2af8b-e8c9-44a7-855e-2d5d1a7db9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:52:31.641480 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2af8b-e8c9-44a7-855e-2d5d1a7db9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:52:31.650457 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2af8b-e8c9-44a7-855e-2d5d1a7db9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:52:31.651858 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2af8b-e8c9-44a7-855e-2d5d1a7db9a9 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:52:31.660976 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1318/2598] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:52:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:52:41.656932 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=6 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:53:04.645118 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d51e8-48c9-4715-a65b-f4e0dfba8fd8 service swift] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:53:04.654834 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d51e8-48c9-4715-a65b-f4e0dfba8fd8 service swift] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:53:04.658743 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1e7bd-4c8a-46d5-b9f2-dfbcec2d98c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:53:04.659338 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1e7bd-4c8a-46d5-b9f2-dfbcec2d98c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:04.659619 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1e7bd-4c8a-46d5-b9f2-dfbcec2d98c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:04.659893 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1e7bd-4c8a-46d5-b9f2-dfbcec2d98c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:04.661629 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-dae1e7bd-4c8a-46d5-b9f2-dfbcec2d98c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:53:04.732796 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1e7bd-4c8a-46d5-b9f2-dfbcec2d98c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:53:04.734252 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1e7bd-4c8a-46d5-b9f2-dfbcec2d98c7 service neutron] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:53:04.745445 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1281/2599] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:53:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:53:05.928841 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2af8b-e8c9-44a7-855e-2d5d1a7db9a9 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:53:05.938075 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2af8b-e8c9-44a7-855e-2d5d1a7db9a9 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:53:05.941611 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db04575738b04f2d95803498e19554c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:53:05.942223 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:05.942498 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:05.942765 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:05.944440 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:53:05.957561 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:53:05.958983 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:53:05.969526 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1319/2600] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:53:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:53:14.740268 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=34 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:53:15.964823 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=6 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:53:47.801789 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043c99-6395-438c-ab11-98c013b7d34b None None] REQUEST_METHOD: `POST` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:47.802857 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043c99-6395-438c-ab11-98c013b7d34b None None] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:47.803012 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043c99-6395-438c-ab11-98c013b7d34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:47.822331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.auth.core [None req-6e043c99-6395-438c-ab11-98c013b7d34b None None] MFA Rules not processed for user `3d937cb6b9054233b587fd1c39a9a662`. Rule list: `[]` (Enabled: `True`). {{(pid=81491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:53:47.823522 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6e043c99-6395-438c-ab11-98c013b7d34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:53:47.830960 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1282/2601] 10.222.0.44 () {58 vars in 967 bytes} [Wed Oct 12 12:53:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:53:47.871582 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:53:47.882975 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd08b93-2ac9-4a86-b6a7-7afdf5c3cf52 service neutron] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:53:47.888227 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de763d-c20e-484a-8650-68365c616866 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36af83e8ee4048d78d2db8784f6347df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:53:47.888821 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de763d-c20e-484a-8650-68365c616866 service swift] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:47.889208 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de763d-c20e-484a-8650-68365c616866 service swift] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:47.889482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de763d-c20e-484a-8650-68365c616866 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:47.897495 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de763d-c20e-484a-8650-68365c616866 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:53:47.899090 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de763d-c20e-484a-8650-68365c616866 service swift] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:53:47.908681 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1320/2602] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Oct 12 12:53:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 12:53:55.793524 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e043c99-6395-438c-ab11-98c013b7d34b None None] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:53:55.795170 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-6e043c99-6395-438c-ab11-98c013b7d34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:53:55.870571 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e043c99-6395-438c-ab11-98c013b7d34b None None] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:53:55.875409 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:53:55.876008 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] REQUEST_METHOD: `GET` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:55.876286 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:55.876556 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:55.878342 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:53:55.889612 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:53:55.891201 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:53:55.901433 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1283/2603] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:53:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:53:55.933120 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de763d-c20e-484a-8650-68365c616866 service swift] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:53:55.999634 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de763d-c20e-484a-8650-68365c616866 service swift] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:53:56.004810 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:53:56.005432 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:56.005714 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:56.006023 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] PATH_INFO: `/v3/users/73179467b36a48a1b1c693dc9c026c9e` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:56.007202 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=73179467b36a48a1b1c693dc9c026c9e)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:53:56.009101 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:53:56.186884 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73179467b36a48a1b1c693dc9c026c9e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.199256 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73179467b36a48a1b1c693dc9c026c9e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.199919 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73179467b36a48a1b1c693dc9c026c9e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.281749 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73179467b36a48a1b1c693dc9c026c9e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.289162 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73179467b36a48a1b1c693dc9c026c9e'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.295218 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1321/2604] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:53:55 2022] DELETE /identity/v3/users/73179467b36a48a1b1c693dc9c026c9e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:53:56.534731 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:53:56.546469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469bb32-4006-4df1-ae14-b6ad6138afd0 service placement] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:53:56.603537 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:53:56.604164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:56.604474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:56.604748 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] PATH_INFO: `/v3/projects/7bc9dba7cc864f998bc613e10cf3b4aa` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:56.605879 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc9dba7cc864f998bc613e10cf3b4aa)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:53:56.607875 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:53:56.752154 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc9dba7cc864f998bc613e10cf3b4aa', 'request_id': 'req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.757712 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc9dba7cc864f998bc613e10cf3b4aa', 'request_id': 'req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.764164 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc9dba7cc864f998bc613e10cf3b4aa', 'request_id': 'req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.851958 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] The token cache is being invalidate because project 7bc9dba7cc864f998bc613e10cf3b4aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:53:56.852588 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.852876 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:56.854439 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1284/2605] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:53:56 2022] DELETE /identity/v3/projects/7bc9dba7cc864f998bc613e10cf3b4aa => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:53:56.858482 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:53:56.859905 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:53:56.873311 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b87da4-d37a-4f6e-874b-0df9ee00fdfc admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:53:56.928779 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:53:56.929388 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:56.929667 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:56.929962 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] PATH_INFO: `/v3/users/7ddc94db34c84a51aa9bacc01972bf60` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:56.931059 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ddc94db34c84a51aa9bacc01972bf60)` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:53:56.932978 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:53:56.938456 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:53:57.102269 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ddc94db34c84a51aa9bacc01972bf60'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.114703 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ddc94db34c84a51aa9bacc01972bf60'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.115372 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ddc94db34c84a51aa9bacc01972bf60'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.150554 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ddc94db34c84a51aa9bacc01972bf60'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.157603 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.notifications [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ddc94db34c84a51aa9bacc01972bf60'} {{(pid=81490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.164087 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1322/2606] 10.222.0.44 () {58 vars in 1224 bytes} [Wed Oct 12 12:53:56 2022] DELETE /identity/v3/users/7ddc94db34c84a51aa9bacc01972bf60 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:53:57.413806 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] Authenticating user token {{(pid=81491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:53:57.415311 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.fernet_utils [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:53:57.427751 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdf1a5-6989-407b-8ef7-6633e8cab1eb admin admin] Validating token access rules against request {{(pid=81491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:53:57.437195 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:53:57.485212 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7cad0b678b4f0182359eae55570531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5655a305f69b4cb0901b114b25e5c478', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:53:57.485906 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:53:57.486204 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:53:57.486474 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] PATH_INFO: `/v3/projects/91fbadf143bb4254bb64423fe3642cdd` {{(pid=81491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:53:57.487575 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91fbadf143bb4254bb64423fe3642cdd)` {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:53:57.489450 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] RBAC: Authorization granted {{(pid=81491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:53:57.654275 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91fbadf143bb4254bb64423fe3642cdd', 'request_id': 'req-60e94834-457f-424a-ad9f-967f6946cc01'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.660129 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91fbadf143bb4254bb64423fe3642cdd', 'request_id': 'req-60e94834-457f-424a-ad9f-967f6946cc01'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.666331 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91fbadf143bb4254bb64423fe3642cdd', 'request_id': 'req-60e94834-457f-424a-ad9f-967f6946cc01'} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.743476 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] The token cache is being invalidate because project 91fbadf143bb4254bb64423fe3642cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:53:57.743762 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.744366 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG keystone.notifications [None req-60e94834-457f-424a-ad9f-967f6946cc01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:53:57.745919 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: [pid: 81491|app: 0|req: 1285/2607] 10.222.0.44 () {58 vars in 1233 bytes} [Wed Oct 12 12:53:57 2022] DELETE /identity/v3/projects/91fbadf143bb4254bb64423fe3642cdd => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:54:07.160001 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:54:07.740469 dv-cinder-812685-16-iscsi devstack@keystone.service[81491]: DEBUG dbcounter [-] [81491] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:56:22.254417 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] Authenticating user token {{(pid=81490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:56:22.255845 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:56:22.393294 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05324a4a-a5e4-45ca-95df-f857546702c8 admin admin] Validating token access rules against request {{(pid=81490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:56:22.451183 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbfa1-8bad-4b11-a399-1042d887969a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f35e7add10924dc49eb9d4a983c89eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e28d11b103f94db0ae7c772a2f086978', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:56:22.451773 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbfa1-8bad-4b11-a399-1042d887969a service placement] REQUEST_METHOD: `GET` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:56:22.452047 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbfa1-8bad-4b11-a399-1042d887969a service placement] SCRIPT_NAME: `/identity` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:56:22.452314 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbfa1-8bad-4b11-a399-1042d887969a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:56:22.454048 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.fernet_utils [None req-51ddbfa1-8bad-4b11-a399-1042d887969a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:56:22.465055 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbfa1-8bad-4b11-a399-1042d887969a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:56:22.467188 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbfa1-8bad-4b11-a399-1042d887969a service placement] RBAC: Authorization granted {{(pid=81490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:56:22.476808 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: [pid: 81490|app: 0|req: 1323/2608] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Oct 12 12:56:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:56:32.472678 dv-cinder-812685-16-iscsi devstack@keystone.service[81490]: DEBUG dbcounter [-] [81490] Writing DB stats keystone:SELECT=89 {{(pid=81490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}